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Chapter 1 
INTRODUCTION 

The intent of the at-risk grants initiated under H.B. 2217 (1988) has been twofold: 1) to 
encourage schools to develop innovat ve strategies for addressing the needs of at-risk students, and 2) to 
integrate those strategies with other programs within the schools. In conjunction with these intentions 
the goal of the evaluation effort has been to determine which strategies have been most successful (i e ' 
"what works" for at-risk youth), and provide feedback about them to ADE and legislators. Over the last 
three years, however, as at-risk programs have become more integrated into total school structures it has 
become increasingly clear to evaluators that "what works" goes beyond the scope of specific at-risk 
program strategies. In fact, what works has to be defined in terms of how it works. Therefore, evaluators 
have had to examine strategies in the context of the total school environment. 

To accomplish this task, the evaluation plan has relied on the premise that understanding program 
processes and the implementation strategies employed by districts is as important as measuring student 
outcomes. The process is as important as the product. Critical considerations included: Who was involved 
in program planning? Did more involvement of staff in program planning result in more effective 
practices in the classroom? How important was open communication in bringing about change** Were 
school/community linkages important in successful programs, and what type of linkages? How did parents 
want to be involved? And, ultimately, what can the state of Arizona do to promote the types of changes 
that will help at-risk students become successful students? With these questions in mind, Morrison 
Institute administered site visits, interviews, surveys of staff and students' perceptions of at-risk programs 
and profiles of student characteristics. All of these combined to "paint the picture" of the social 
emotional, and educational environment in which Arizona's at-risk children live and learn. 

Merely increasing our understanding of the change process and the at-risk program context 
however, is not meaningful in and of itself. Rather, what is learned must be applied to determining how 
an education system can be made to be more responsive to the needs of at-risk students. Therefore 
evaluators have looked at several indicates reiated to student success: school attendance, promotion to 
the next grade, sense of worth, credits earned, and academic achievement. Determining the degree to 
which these student outcomes were achieved has become a ,-oaJ of the evaluation effort. 

This project report aggregates data for all 55 projects (42 K-3; 13 7-12). For the purposes of the 
report, however, the aggregate data have been analyzed to reveal patterns that apply across programs. 
Trends over the three years of the project are shown for certain indicators. Additionally FY 1990-91 
evaluation data were disaggregated by region (urban/suburban, rural, and reservation) and by phase (I 
and II for K-3 only) in order to examine the hypothesis that "what works" might vary according to these 
distinctions. In fact, there are neteworthy differences among progiams which will be discussed throughout 
this report. 

Table 1-1 and Figure 1-1 on the following pages show the distribution of the 55 programs by 
region and phase as well as by their geographical locations. Subsequent chapters in this report present 
detailed information about evaluation methodology, the K-3 and 7-12 programs, program budgets and 
conclusions. A companion report, the Arizona At-Risk Pilot Project Policy Report, discusses policy issues 
gleaned from the research that are considered essential for developing state level policies relevant to 
educational programming and funding for at-risk youth. 
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Another report on the Arizona At-Risk Pilot Project is forthcoming in June of 1992~at the 
conclusion of the fourth year of the project. This repoit will consist of an in-depth look at successful 
practices implemented by specific at-risk pilot districts, including student-oriented educational and support 
programs, parent involvement activities, and staff training. This repoit, entitled "Promising Practices for 
At-Risk Youth, H is intended to provide educators with blueprints for replicating successful processes, 
strategies, and outcomes. 



Table 1-1 





ARIZONA AT-RISK PILOT PROGRAMS 




Urban/Suburban 


Rural 


j Reservation * 


K-3 

• Phase I 

(Be/?an FY 


Crcighton Elementary 
Laveen Elementary 
Littleton Elementary 
Murphy Elementary 
Oiborn Elementary 
Phoenix Elementary 
Roosevelt Elementary 
Wilson Elementary 


Ash Fork Unified 
Coolidge Elementary 
Mary C. O'Brien Elementary 
Morristown Elementary 
Nogales * Unified 
Picacho Elementary 
Somerton Elementary 


Chinle Unified 
Ganado Unified 
Kayenta Unified 
Page Unified 
San Carlos Unified 
Sanders Unified 
Whiteriver Unified 


• Phase II 

(Began FY 
1989-90) 


Avondale Elementary 
Isaac Elementary 
Balsz School** (Balsz) 
El Mirage School** (Dysait) 
Los Panchitos School** 

(Sunnyside) 
Scales School** (Tcmpc) 


Aguila Elementary 
Buckeye Elementary 
Douglas Elementary 
Eloy Elementary 
Gadsden Elementary 
Hyder Elementary 
Salome Consolidated 
Stanfield Elementary 


Fort Thomas Unified 
Holbrook Unified 
Red Mesa Unified 
Peach Springs School** 

(Peach Springs) 
Cameron School** 

(Tuba City) 
Gap School** (Tuba City) 


7-12 

• Phase I 

(Began FY 
1988-89) 


Crcighton Elementary (7-8) 
Dysart Unified 
Sunnyside Unified 
Tucson Unified 


Nogales Unified 

Pinal County Consortium 

• Apache Junction Unified 

• Casa Grande Elem. (7-8) 

• Casa Grande UHS 

• Central AZ. Alternative 

• Coolidge Unified 

• Mammoth-San Manuci 
Unified 

• Maricopa Unified 

• Santo Cruz Valley UHS 

• Superior Unified 
Somerton Elementary (7-8) 


Ganado Unified 
Kayenta Unified 
San Carlos Unified (7-8) 
Sanders Unified 


• Phase II 

(Began FY 
1989-90) 


Pima Co. Detention Center 


Manna Unified 




* Theie district* are located on rciervations or are in locations with Native American student populations of 
approximately 50%. 

** These are school-based project*. Districts are indicated in parentheses. 



NOTE: Urban/suburban programs are referred to as "urban" throughout this report. 
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Chapter 2 
EVALUATION METHODOLOGY 



This chapter provides an overview of the methodology employed in the Arizona At-Risk Pilot 
Project during FY 1990-91. While it includes some retrospective information, it does not repeat prior 
discussions of project methodology documenting the specifics of previous years' efforts. For those, the 
reader is referred to the Arizona At-Risk Pilot Project FY 1989-90 Project Report (Bierlein, Vandegrift, 
Hartwell, Sandler & Champagne, 1990) as a supplement to this document. 



DESIGN OVERVIEW 

In June of 1989, after the first year of the pilot project (FY 1988-89) 2 , Morrison Institute was 
commissioned by the Arizona Department of Education to design and implement a comprehensive project 
evaluation. A "design team 1 ' of evaluation and assessment experts scrutinized the feasibility of conducting 
a true experimental or quasi-experimental study comparing program and non-program participants. This 
team dismissed a control group study as unrealistic, primarily because of the magnitude of the project 
(33 "phase F sites already in operation and 22 "phase II" sites approved for FY 1989-90) and projected 
difficulties acquiring meaningful comparative data from non-funded control sites. 

Perhaps what is more importai t is that the team decided that determining the "relative worth" of 
at-risk programming was not the most critical issue at stake for the evaluation. Rather, based on 
demographic projections, exhaustive reviews of articles and research reports, and expert opinion that 
posited "at-riskness" as an endemic educational problem, the value of at-risk programs was accepted as 
a given. Evaluation efforts, therefore, focused on determining what strategies, among the many being 
implemented at the 55 pilot sites, hold most promise for Arizona's at-risk youth~in short, "what works." 
Attention was also focused on examining the impact of state policy on the development and 
implementation of programs over the four years prescribed by H.B. 2217 (1988). 

The overall evaluation design adopted a long-term, in-depth approach in order to generate detailed 
descriptions of program implementation processes, services, and outcomes. This design integrated the use 
of both qualitative and quantitative methods. Data were collected via site visits incorporating observation 
and interviewing, district self-reported program information (including participation data, self-evaluation 
studies, cohort data, retention rate information for K-3 sites, and budget information), and surveys 
administered to teachers and grade 7-12 students. In addition, narrative report guidelines, standard 
interview "protocols," surveys, and numerous standard reporting forms were developed and refined and 
regular training sessions were held with both site directors and evaluators. 

The question of "what works" was addressed using cross-site analyses through which patterns of 
program similarities and differences were identified— in clientele, settings, strategies, and implementation. 
As patterns emerged, they formed a basis for aligning reports, interviews, surveys, and forms. The 



While funding wu appropriated and allocated to district* during 1988, most programs were only in operation for a maximum 
of one semester (January - June 1989). 
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alignment generated multiple sources of data that were triangulated 3 to discern areas of consensus 
regarding "what wor j" in terms of such issues as: which programs reach the most students, parents, 
and/or staff; which programs prompt organizational change (e.g., "systemic reform"); and, which 
practices are viewed as successful through the eyes of students, teachers, administrators, counselors, 
aides, parents, and community members. 

In conjunction with descriptive databases, program evaluation efforts included a "cohort study" 
in which academic histories for a randomly selected group of student participants were tracked over time. 
This tracking effort, essentially mandated by H.B. 2217 (1988), was based on an assumption that "what 
works" might be defined in terms of objective evidence of positive change in test scores, absenteeism, 
rates of failure, credits earned, and other recognized indicators of success. 

Superimposed on the question of "what works" was the additional question: "For whom?" This, 
in turn, prompted other questions that figured prominently in data collection and analyses, such as: How 
comparable are at-risk students? Do different types of students benefit from different kinds of services? 
What works for parents in terms of increasing their involvement? What works for staff in terms of 
developing the skills needed to work with at-risk students and their parents? 

Quantitative databases (e.g. , cohort study) were designed with parameters and constraints relevant 
to more rigorous research designs (e.g., reliability, validity). Qualitative databases were designed keeping 
in mind "criteria of soundness [that] more accurately reflect the assumptions of the qualitative paradigm" 
including credibility, transferability, dependability, and confirmability (Marshall and Rossman, 1989, pp. 
144-145). Each database has its own merits, caveats, and limitations. Also each database is only one of 
many used in defining "what works"; no single data set fidly describes the phenomena being examined. 
Taken as a whole, however, trends emerge regarding programs and activities that hold promise for at-risk 
youth in Arizona and the policy issues that need to be addressed to enable such programs to flourish. 

As a result of the study's design and data collection efforts outlined above, the following 
databases were compiled during FY 1990-91: 

1) student profile data completed by school personnel for 3,618 at-risk K-3 students and by 1,627 
at-risk 7-12 students; 

2) descriptive narrative data compiled by individual site evaluators (phase I) and district personnel 
(phase II) for each of the 55 individual programs; 

3) participation data for students, parents, and staff reported by district personnel and confirmed by 
Morrison Institute staff; 

4) survey data (e.g., mean ratings) obtained from 1,021 K-3 teachers and 982 7-12 staff members; 

5) open-ended question responses regarding program strengths and recommendations for change 
reflecting 4,042 comments from K-3 teachers and 1,041 comments from 7-12 staff members; 



' Ai described by Marshall and Rossman (1989), triangulation ii one strategy to enhance a qualitative study's credibility and 
generalizability. It is "... the act of bringing more than one source of data to bear on a single point [such that] data from different 
sources can be used to corroborate, elaborate, or illuminate the research in question* (pp. 146; cf. Rossman and Wilson 1985). 



6) open-ended question responses regarding program outcomes obtained from 986 K-3 teachers and 
843 7-12 staff members; 

7) survey data obtained from 1,627 7-12 students; 

8) structured interview data collected by the site evaluators encompassing 748 interviews involving 
153 parents, 460 school staff (i.e., 220 teachers, 127 administrators, and 113 specialists/support 
staff), 17 school board members, 18 community members, and 100 students in grades 7-12; 

9) demographic and "impact" data (e.g., achievement test scores) following up on 3,958 K-3 
students and 1,305 7-12 students who are being tracked longitudinally in the cohort study; 

10) information regarding retention (e.g., rates and policies) for 41 of the 42 K-3 programs; 

11) budget information for 53 of the 55 pilot demonstration projects. 



DATA SETS AND ANALYSES 

STUDENT PROFILES 

The K-3 Student Profile consisted of 30 multiple choice questions, ten that were demographic in 
nature and 20 that focused on factors often associated with "at-risk status.* 1 The student profile was to 
be completed hy the school person who knew the child best; in most cases, this was the child's primary 
teacher. Districts were asked to profile all or a sample of up to 100 at-risk students served by their 
programs. Descriptive statistics were computed using SAS Software for Di % a Analysis for all questions 
for all students, students by phase (I or II), students by region (urban, rural, or reservation), and students 
by phase and region. Certain items were cross-tabulated to reveal trends of indicators in relation to 
academic performance by phase, region, and phase by region. 

The 7-72 Student Profile consisted of 30 questions, 11 that were demographic in nature and 19 
that focused on factors often associated with " at-risk status/ Districts were asked to administer the survey 
to all or a sample of at-risk students in each one of their program components. Descriptive statistics for 
all questions were computed using SAS Software for Data Analysis for all students and for students by 
region and by grade level. Similar to analyses of K-3 data, certain items were cross-tabulated with 
students' self-ratings of academic performance for students as a whole, by region, and by grade level (7-8 
or 9-12). Data were not disaggregated by phase (as there are only two phase II districts) or region by 
grade (due to small sample sizes at this level of detail). 



PROGRAM NARRATIVES 

Descriptive data were generated bi-annually (fall and spring) by a team of site evaluators for each 
of the 33 phase I sites, and annually (spring) by district personnel for each of the 22 phase II sites. Since 
the summer of 1989, a series of five reports 4 per phase I district and two reports per phase II district 
have been generated. Both phase I and phase II site narratives were prepared following model reports and 



4 The first of the five reports was district-generated but edited and formatted by Morrison Institute staff. These reports 
summarized program activities which occurred during FY 1988-89, the first year of the four-year longitudinal study. 
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guidelines to ensure that certain common program features were addressed across sites, while allowing 
unique aspects of each program to be discussed. Reports have varied somewhat over time. For example, 
initial program descriptions encompassed discussions of planning, needs assessments, and so forth, that 
were not addressed in subsequent reports. Moreover, more detailed descriptions of program services have 
evolved oyer time in light of site evaluators* increased understanding. To help ensure the credibility and 
confirmability of descriptive data, phase I district personnel were provided opportunities to review draft 
versions of reports and/or provide written addenda; phase II reports were verified through on-site visits 
by site evaluators. 

Descriptive data were analyzed in a number of ways (e.g., to identify intervention strategies for 
at-risk youth and their parents, to quantify administrative turnover across programs, to assess common 
areas of program strengths and weaknesses). In general, such analyses were initially conducted by one 
or two Morrison Institute analysts and presented back to district personnel and/or Morrison Institute staff 
for discussion, verification, and/or revision. For example, during the summer of 1990, all district 
proposals and Morrison Institute formative and summative reports were analyzed to identify and classify 
student, parent, and staff development activities that are funded and/or formally integrated as part of the 
at-risk programs. As refined in consultation with ADE and district personnel, this analysis resulted in the 
development of a list of student service strategies, parent involvement strategies, and staff development 
strategies used in structuring evaluation efforts. 

DISTRICT-REPORTED PARTICIPATION 

A review and analysis of FY 1989-90 district-submitted participation data prompted the 
development of a more structured approach to collecting participation data for FY 1990-91. Lists of 
student, parent, and staff development strategies (as described in the previous paragraph) provided the 
structure for new participation reporting forms. For 1990-91, K-3 districts were asked to report all 
student services delivered through their programs; 7-12 districts were asked to report all student activities 
delivered by each program component, and to report participation as a result of program linkages with 
other district-sponsored initiatives and community-based organizations. Several measure were also taken 
in an effort to ensure the comparability of data across projects. 

Fall participation data, submitted by the districts in February 1991, revealed a number of errors 
and discrepancies-some made by the districts and others due to problems with the data-collection forms. 
Modifications were made by research analysts at Morrison, and the fall data were returned to the districts 
for corrections and clarification. Spring participation data submitted in June 199 1 revealed fewer reporting 
errors but still required verification by Morr^on Institute staff. This report contains only participation 
numbers reflecting "official" program activities and not those for other (i.e., non-program) district 
activities, even though some districts included data for the latter type. 

Using Lotus spreadsheets, data were tabulated by semester and by year, with yearly averages. 
Data were disaggregated by selected variables (e.g., phase and /or region). 

TEACHER SURVEYS 

Both the Arizona At-Risk Pilot Project: K-3 Teacher Survey and 7-12 Teacher Survey consisted 
of 60 multiple-choice questions. Five of the K-3 survey questions and four of the 7-12 survey questions 
were demographic in nature. The remainder of the items on each survey focused on school/district 
intervention strategies for students, parents and staff; outcomes; and factors often associated with 
successful program implementation. Each of the 55 pilot demonstration projects provided Morrison 
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Institute with the estimated number of staff at the respective school or district. Using a table for 
determining needed sample sizes for an analysis with a 95 percent level of confidence (Krejcie & Morgan, 
1981), Morrison Institute provided each district with the number of surveys needed to complete such an 
analysis. Districts with very large numbers of teachers (i.e., J> 100) were directed to randomly sample 
their population. 

Using SAS Software for Data Analysis, frequency distributions were tabulated for demographic 
questions. Teachers rated the remaining items on a Likert scale. Mean scores were calculated for each 
item and examined with respect to selected respondent variables using Chi-square tests of significance. 
One variable used for 7-12 analyses concerned the respondent's level of program awareness: program 
staff members versus non-program staff members who were either aware or unaware of the district's pilot 
at-risk program. This breakdown, prompted through discussions with 7-12 program staff, was included 
because many 7-12 programs are self-contained and vary in their degree of integration with other school 
services. A "program awareness" variable did not appear as an issue with respect to K-3 programs. 
However, this may be a variable worth exploring among K-3 school staff in the future. 

In addition to the 60 multiple-choice questions, teachers were given several open-ended questions 
regarding program strengths, recommendations for change, and selected program outcomes. Questions 
were analyzed by category (e.g., student services) and/or selected respondent variables. 

INTERVIEWS 

Structured interviews conducted with school personnel, parents, community members, and 7-12 
pilot program students followed interview "protocols" designed by Morrison Institute staff. This was to 
ensure that the 1 1 site evaluators focuse! on similar issues for each of the programs. However, following 
ethnographic interviewing techniques such as those described in Spradley (1979), site evaluators were 
encouraged to follow-up "standard" questions with more in-depth questions as relevant. Questions focused 
primarily on most and least effective services for students, parents, and staff and on the integration of 
services within the school and with the community. Program changes and recommendations were also 
elicited. Site evaluators preserved the confidentiality of their interviewees in an effort to ensure integrity 
of the data. 

Interview data were analyzed across several categories to determine commonalities and 
discrepancies. Themes related to these factors were then identified. To prevent any bias in the analysis 
based on findings from other databases, interview data were analyzed by an evaluator who had no 
knowledge of results from those other databases. Appendix A, Table A-9 portrays the distribution of 
personnel interviewed for K-3. Interview data for 7-12 programs are shown in Appendix B, Table B-8. 

COHORT STUDY 

During FY 1989-90, districts were asked to identify all of their at-** ! sk students if their total 
population was sufficiently small or a sample of their at-risk populations up to 100 students per grade. 
Using district-submitted lists of students, Morrison Institute randomly selected students for inclusion in 
the cohort study. Districts then assigned each student included in the study a nine-digit identification code, 
preferably the student's social security number. Specially designed "cohort coding sheets" were used to 
collect a variety or demographic, programmatic, and "impact" data for each student identified. 
Information requested was not only for FY 1989-90, but also for FY 1988-89. Data reported were 
analyzed using SAS Software for Data Analysis. Aggregate data were prepared for the 1989-90 cohort; 
trend analyses were conducted for those students for whom 1988-89 and 1989-90 data were available. 




Based on district feedback and FY 1989-90 analyses, the cohort coding sheets were modified for 
FY 1990-91. Districts were again asked to complete demographic, programmatic, and impact data for 
those students originally identified in FY 1989-90. Analyses for FY 1990-91 (year three of the study) are 
primarily based upon students in the original cohort group for whom three years of complete data are 
available. Additional analyses were conducted using subsets of information from the cohort group. 

RETENTION INFORMATION K-3 

During spring 1991, each of the 42 pilot project districts received a "retention rate reporting 
form" that was developed by Morrison Institute in conjunction with the Arizona K-3 Advisory 
Committee/ At-Risk Subcommittee. At-risk project directors or their designees were asked to respond to 
several questions about retention criteria and policies in the school and/or district. Respondents were 
encouraged to provide written documentation to support their responses whenever possible, and were also 
asked to provide detailed retention data for their district's entire K-3 population. Every effort was made 
to identify inconsistencies and mathematical errors and to provide the most accurate information. 

Forty-one districts (representing the 42 programs) were given the retention rate forms; 40 
responded representing all 22 phase I programs and 19 phase II programs. Retention information and 
rates were analyzed by phase, region, and phase by region. Additionally, retention rates for at-risk pilot 
sites were compared with district-level data that included retention rates from non-participating sites. 

STUDENT SURVEYS M2 

The 7-72 Attitude Survey consisted of 30 items. Sixteen items focused on student satisfaction with 
instructional, vocational, and support services received throughout the FY 1990-91 school year. Fourteen 
questions focused specifically on at-risk pilot program outcomes with respect to specific program 
components. In order for students to be able to rate the district's program components, the last page of 
each survey was "customized" using the name of the component the student was to rate (e.g., Alternative 
School; Mr. Smith's class). 

Districts were asked to administer the survey to all, or a sample of, at-risk students served by 
their programs. Descriptive statistics were computed using SAS Software for Data Analysis. Ratings for 
each program service and outcome were generated according to selected student variables. Chi-square 
tests of significance were also conducted. 

BUDGET INFORMATION 

Arizona S.B. 1079 (1991) established a joint legislative committee to study at-risk funding and 
programs for at-risk pupils and extended the at-risk pilot programs founded under H.B. 2217 (1988) for 
one additional year (through FY 1992-93). S.B. 1079 (1991) also specified an examination of amounts 
spent by these programs. Morrison Institute, in consultation with ADE and the state K-3 Advisory 
Committee, developed the FY 1990/91 At-Risk Program Grant Breakdown form requiring that districts 
prorate their at-risk budgets by function including: district-level and school-level administration, direct 
student services, parent/community outreach activities, staff development and training, program 
evaluation, and indirect costs. In addition, districts were to report any unspent monies for FY 1990-91. 

The budget form was provided to all 55 programs (42 K-3; 13 7-12). Fifty-three districts 
responded, representing 40 K-3 programs and all 13 7-12 programs. Of the two K-3 districts excluded 
from the analysis, one district's report was received well after the submission deadline and after analyses 
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were completed; one district did not submit the information. Efforts were made to obtain the most 
accurate information in all cases. Data were analyzed separately for K-3 and 7-12 programs. Program 
cost-per-pupil figures were calculated for each district and were examined with respect to several key 
variables (e.g., size of district, region). Appendix C presents supplemental data pertaining to budget 
analyses. 



A NOTE ON THE VALIDITY OF THE DATA 

The overall research design, as noted, is conceptually qualitative and particularly subject to 
scrutiny relative to four criteria: credibility, transferability, dependability, and confirmability. The 
evaluation team continuously subjected the methods and findings of the study to internal and external 
review in their endeavor to protect its integrity with respect to these criteria. Wj are, therefore, confident 
that these efforts have produced important and meaningful information. With respect to the quantitative 
databases compiled, sampling methods and analysis techniques were employed in recognition of key 
variables potentially affecting the reliability and validity of the findings. Morrison Institute evaluators are 
aware of specific limitations associated with all databases (see Appendix D); the most pertinent of these 
will be referenced in relation to the analyses presented in subsequent discussions. 

In sum, the study involved collection of a large amount of data and information, both quantitative 
and qualitative 5 . The data sources included self-reports, individual judgments and perceptions, school 
records, and other items that are subjective in nature and that may reflect bias or contain errors of fact. 
These types of data sources are essential for a comprehensive evaluation but they must be interpreted with 
considerable caution. Consequently, the evaluation team exercised great care in attempting to confirm the 
accuracy of data while searching for patterns and consensus across related databases. The strength of this 
evaluation study rests not only in its breadth but in its depth. 



hundreds of data table* were generated from the analyse* of all the data acta. Analyiea were done by district, by region, by 
phaac (K-3 only) and by grade level (7-12 only). Any data from analyses referenced in this report that is not included in the text 
or the appendixes is available upon request. 
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Chapter 3 

K-3 AT-RISK PROGRAMS; DESCRIPTIONS AND EVALUATION RESULTS 



This chapter synthesizes information from each database compiled for the analysis of the K-3 at- 
risk programs. After evaluation findings have been presented, the final section will discuss conclusions 
drawn as a result of the analysis. 



DESCRIPTION OF THE K-3 STUDENTS: TEACHER PERSPECTIVES 

In order to assess whether or not at-risk programs were meeting the needs of at-risk students, 
Morrison Institute sought to better understand these needs. Therefore, descriptive data were collected 
regarding specific factors that place K-3 students at risk. Ideally, this data would have been obtained from 
standardized databases at the local level, but since such data are largely inaccessible or unavailable, 
students were profiled, instead, by district personnel using the K-3 Student Profile (cf. Chapter 2). 

K-3 AT-RISK STUDENT AND FAMILY CHARACTERISTICS 

A two-year comparison of teachers' perceptions of at-risk students' family characteristics was 
conducted to determine whether or not there were annual trends in K-3 student profiles. Although the 
comparison reveals some differences, key demographic data show that teachers' views of their student 
populations were fairly reliable over time. Teachers both years, for example, reported that a majority of 
students lived with both parents (o5 percent in 1989-90; 55 percent in 1990-91). Although many teachers 
did not respond to items related to parent education and employment, when this information was known, 
teachers reported that most parents/guardians had a high school diploma or less, that mothers/female 
guardians were largely not employed outside the home, and that most fathers/male guardians occupied 
"blue collar" jobs. 

FY 1990-91 student profile data were disaggregated by region (urban versus rural versus 
reservation) and by phase (I versus II). Phase I versus II analyses were conducted in response to the 
notion that program differences might occur in relation to the length of time the programs had been in 
operation. While phase differences have been found in other databases, comparisons of student profile 
data revealed few noteworthy demographic differences befcuen phase I and phase II „ites. 

Regional findings show that urban students made up ?pproximately 37 percent of the 3,618 
students profiled, while rural students comprised 32 percent and reservation students 31 percent of thj 
total. Students profiled were similar in all three regions in relation to gender, grade distribution, and 
achievement levels. A key exception was that reservation programs, as expected, were primarily serving 
Native American children (Table 3-1). 

Perhaps the most noteworthy information depicted in Table 3-1 relates to the category "Academic 
Rating." Data suggest that not all at-risk youngsters were necessarily at risk because they were low 
achievers. In each region, roughly one out of five students was rated as achieving satisfactorily (i.e., in 
the high-very high range), yet was considered to be "at-risk." 



Table 3-1 



COMPARISON OF 1990-91 K-3 AT-RISK STUDENT CHARACTERISTICS BY REGION 






(Urban N == 


1336; Rural N ■ 


= 1158; Reservation N = 1124) 






Gender 


Male 


Female 




No Reaponse 




Urban 


55% 


45% 




<1% 




Kurai 




*fO To 




<1% 




Reservation 


55% 


45% 




<1% 




Ethnicity 


While 


Black 


Hispanic 


Native 


No Response 










American 






Urban 


20% 


11% 




5% 


2% 


Rural 


25% 


5% 


64% 


4% 


2% 


Reservation 


6% 


<1% 


2% 


92% 


1% 


Grade Level 


Kindergarten 


First 


Second 


Third 


Multi-Grade 


No 














Reaponse 


Urban 


31% 


26% 


2z% 


19% 


2% 


<1% 


Rural 


22% 


27% 


23% 


25% 


4% 


<1% 


Reservation 


19% 


27% 


23% 


25% 


10% 


<1% 


Academic Rating 


Very Low 


Low 


Average 


High 


Very High 


No 














Response 


Urban 


17% 


28% 


33% 


15% 


6% 


<1% 




18% 


31% 


32% 


13% 


6% 


<1% 


Reservation 


15% 


29% 


38% 


13% 


5% 


<1% 



In addition to the information depicted in Table 3-1, the K-3 Student Profile was used to gather 
information regarding students' families. As shown in Table 3-2, family profiles pointed out some 
regional differences in teachers' perceptions of students' life circumstances. On the whole, more urban 
children were reported to be living in "broken homes" (i.e., single parent households or with a 
stepparent) than either rural or reservation children. More reservation children, however, were, reported 
to live with people other than their parents. 

A greater number of reservation parents (or guardians) were perceived as ha\ .ng some college- 
more than in either the rural or urban areas. At the same time, the percentage of unemployed 
parents/guardians for the reservations was cumulatively (i.e., mothers + fathers) higher than in either 
of the other two regions. Finally, the number of monolingual speakers of languages other than English 
was perceived to be higher for rural and urban areas than for reservation areas, but reservation parents 
were reported to be more proficient bilingually-a characteristic that supports the perception of their 
higher levels of education. 
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Table 3-2 



COMPARISON OF 1990-91 FAMILY PROFILE FOR K-3 AT-RISK STUDENTS BY REGION 




(Urban N = 1336; Rural N = 1158; Reservation N ■ 


1124) 




1 

| Family Structure 


Mother/Father 


Natural Parent/ 


| 

Single Parent 


Other 


No Response 






Stepparent 






Urban 


50% 


10% 


30% 


4% 


5% 


Rural 


59% 


10% 


23% 


5% 


3% 


Reservation 


57% 


8% 


21% 


9% 


5% 


Parent Education 


< Hig.i School 


High School 


Some College 


College 


Don't Know/ 










f~\ , _ J., a 4. _ 

or&uuate 


No Response 


• Mother 












Urban 


27% 


18% 


4% 


1% 


50% 


Kunu 


33% 


23% 


4% 


2% 


38% 


Reservation 


20% 


29% 


10% 


2% 


39% 


• Father 












Urban 


ID Ho 




i at 

3% 


1% 


66% 


Rural 


27% 


18% 


3% 


2% 




Rescrvauon 


15% 


24% 


7% 


2% 


52% 


rorcni vyceupauun 


Not Employed 


Laborer/ 


Agricultural 


Professional 


Don't Know/ 






Clerical 






No Response 


• Mother 












Urban 


42% 




1% 


J nf 


24% 


Rural 


53% 




2% 


3% 


12% 


Reservation 


50% 


30% 


<1% 


6% 


14% 


• Father 












Urban 


5% 


37% 


3% 


6% 


52% 


Rural 


7% 


44% 


12% 


4% 


34% 


Reservation 


19% 


39% 


3% 


3% 


33% 


Parent Language Skills 


Monolingual 


Limited English 


Proficient 


Monolingual 


Other/No 


(1990-91 survey only) 


(other than 


Proficient 


Bilingual 


English 


Response 




English) 


(LEP) 






Urban 


18% 


15% 


19% 


43% 


6% 


Rural 


30% 


14% 


15% 


36% 


4% 


Reservation 


5% 


32% 


38% 


16% 


8% 



PATTERNS OF AT-RISK INDICATORS AND ACHIEVEMENT 

The K-3 Student Profile also was used to gather information regarding "at-risk indicators" (i.e., 
factors identified in the literature as contributing to a child being considered at risk of academic failure). 
These indicators were analyzed in several ways. First, data were examined to identify indicators that 
occurred with the greatest frequency. Second, indicators were identified that were perceived to negatively 
affect the academic performance of K-3 students. Third, indicators were examined with respect to their 
degree of negative effect. Fourth, indicators contributing to "at-riskness" were identified across levels 
of achievement, as well as within levels. All analyses were conducted for the total profiled population 
as well as for each region. Appendix A (Table A-l) presents the raw data used in these analyses. 
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How do teachers describe at-risk students? 



Table 3-3 depicts percentages of the K-3 student population to whom various at-risk indicators 
applied, regardless of whether or not the indicator was perceived as negatively influencing academic 
achievement. This table shows that all three regions share the same "top nine" most frequently occurring 
indicators (although they occur in different orders). Rank ordered by "TOTAL" frequency, the top 
indicators of "at-riskness" are: 



• No/little parent participation in school-related activities 59% 

• Low annual income (less than $15,000/family) 58% 

• Few educational/reading materials around the home 56% 

• Negative to inconsistent self-esteem of the child 50% 

• No/little parent support for child's education 49% 

• Difficulties maintaining parent-school communication 47% 

• Substandard living conditions (e.g., homes without plumbing or electricity year-round) 44% 

• Low levels of English proficiency 42% 

• Emotional/behavioral problems exhibited by the child (e.g., stealing) 33% 



Table 3-3 



% K-3 POPULATION TO WHOM AT-RISK INDICATORS APPLY | 


URBAN 




RURAL 




RESERVATION 




TOTAL 




(n = 1336) 




(n = 1 158) 




(n = 1124) 




(n = 3618) 




Low annual income 


59 


Few reading materials 


61 


Low par. participation 


68 


Low par. participation 


59 


Low par. participation 


57 


Low annual income 


60 


Low parent support 


61 


Low annual income 


58 


Few reading materials 


52 


Low par. participation 


54 


Poor comm. w /p&rtntj 


57 


Few reading materials 


56 


Low self-esteem 


49 


Low self-esteem 


46 


Few reading materials 


56 


Low self-esteem 


50 


Low parent aupport 


44 


Low Eng. proficiency 


44 


Low self-esteem 


55 


Low parent support 


49 


Poor comm. w/parcnts 


42 


Poor comm. w/parents 


43 


Low annual income 


54 


Poor comm. w/parents 


47 


Substandard home 


42 


Low parent support 


42 


Low Eng. proficiency 


52 


Substandard home 


44 


Low Eng. proficiency 


32 


Substandard home 


43 


Substandard home 


47 


Low Eng. proficiency 


42 


Emot/beh problems 


31 


Emot/beh problems 


31 


Emot/beh problems 


39 


Emot/beh problems 


33 


>2 schools attended 


31 


>2 schools attended 


28 


Sub. abuse by parents 


29 


_> 2 schools attended 


26 


Transience/mobility 


20 


Transience/mobility 


19 


"Latch-key" situation 


26 


Health problems 


19 


"Latch-key" situation 


19 


Recent immigration 


18 


Sibling caregiver 


23 


Sub. abuse by parents 


19 


Health problems 


18 


Abusive home 


18 


Health problems 


22 


"Latch-key" situation 


19 


Retained >_ 1 time 


16 


Health problems 


17 


Abusive home 


21 


Abusive home 


18 


Sibling caregiver 


15 


"Latch-key" situation 


16 


Retained 2. 1 


21 


Sibling caregiver 


18 


Abusive home 


15 


Sub. abuse by parents 


15 


J>2 schools attended 


17 


Transience/nability 


17 


Sub. abuse by parents 


14 


Sibling caregiver 


14 


Transience/mobility 


12 


Retained >. 1 time 


16 


Recent immigration 


12 


Retained 3» 1 time 


13 


Sibling dropout(a) 


11 


Recent immigration 


12 


Sibling dropout(s) 


6 


Sibling dropout(s) 


9 


Sub. abuse by child 


10 


Sibling dropout(a) 


9 


Sub. abuse by child 


2 


Sub. abuse by child 


4 


Recent immigration 


4 


Sub. abuse by child 


5 



Further analysis of Table 3-3 suggests that more at-risk factors are associated with reservation 
children than with children in other regions. This finding is supported by data showing that 15 of the 20 
at-risk indicators applied to higher percentages of reservation students than to urban or rural students. In 
comparison, rural students rated higher for only three indicators (i.e., low family income, inadequate 
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educational/reading materials, and immigration), and urban students rated higher for only two (i.e., J> 
2 elementary schools attended and transience/mobility). 

Which indicators affect K-3 student achievement? 

Not all indicators that applied to individual children were perceived as contributing to low 
achievement. Table 3-4 presents the percentage of children believed to be adversely affected by each 
indicator. Not surprisingly, Table 3-4 indicates that the nine most frequently occurring indicators also are 
the nine indicators perceived as negatively influencing the most children. Regional differences may be 
discerned, however, in the rank order of these indicators. For example, the greatest number of urban and 
rural children were felt to be affected by a lack of educational or reading materials and low annual 
incomes, while more reservation children were reported to be affected by a lack of parent participation 
and low self-esteem. 



Table 3-4 



% K-3 POPULATION WHOSE ACHIEVEMENT IS ADVERSELY AFFECTED 


URBAN 
(n = 1336) 


RURAL 
(n = 1158) 


RESERVATION 
(n = 1 124) 


TOTAL 
(n = 3618) 


Few reading materials 48 
Low annual income 46 
Low par. participation 45 
Low self-esteem 44 
Low parent support 38 
Poor comm. w/parenU 36 
Substandard home 34 
Emot/beh problems 28 
Low Eng. proficiency 24 


Few reading material* 51 
Low annual income 4 1 
Low par. participation 40 
Low self-esteem 38 
Low Eng. proficiency 34 
Poor comm. w/parents 34 
Low parent support 33 
Substandard home 31 
Emot/beh problems 25 


Low par. participation 46 
Low self -ester, fi 45 
Low parent support 44 
Few reading materials 43 
Low Eng. proficiency 38 
Poor comm. w/parents 37 
Low annual income 32 
Emot/beh problems 30 
Substandard home 29 


Few reading materials 47 
Low par. participation 44 
Low self-esteem 43 
Low annual income 40 
Low parent support 38 
Poor comm. w/parents 36 
Lew Eng. proficiency 32 
Substandard home 31 
Emot/beh problems 28 


>2 schools attended 23 
Transience/mobility 17 
Health problems 15 
Abusive home 14 
Sub. abuse by parents 13 
"Latch-key- situation 12 
Sibling caregiver 1 1 
Recent immigration 10 
Retained >. 1 time 8 
Sibling dropout(s) 5 
Sub. abuse by child 1 


>2 schoc's attended 17 
Transience/mobility 1 5 
Recent immigration 13 
Abusive home 13 
Health probldiu 13 
Sub. abuse by parents 10 
"Latch-key" situation 10 
Sibling caregiver 7 
Retained >_ 1 time 6 
Sibling dropout(s) 4 
Sub. abuse by child < 1 


Sub. abuse by parents 21 
Abusive home 17 
"Latch-key" situation 16 
Health problems 15 
Sibling caregiver \2 
Retained J> 1 time 9 
_>2 schools attended 8 
Sibling dropout(s) 6 
Transience/mobility 6 
Sub. abuse by child 6 
Recent immigration < 1 


_> 2 schools attended 17 
Sub. abuse by parents 15 
Health problems 14 
Abusive home 14 
"Latch-key" situation 13 
Transience/mobility 1 2 
Sibling caregiver 10 
Retained >_ 1 time 8 
Recent immigration 8 
Sibling dropout(s) 5 
Sub. abuse by child 2 



In accordance with previous conclusions, Table 3-4 indicates that more reservation children are 
adversely affected by at-risk factors than are children in urban or rural areas. Thirteen of 20 indicators 
were rated as affecting a higher percentage of reservation children than others. In contrast, only five 
indicators reportedly affe( id more children in u»tan areas (multiple school attendance, mobility, 
substandard living conditions, low family income, and health problems), and only two indicators affected 
more children in rural areas (inadequate educational/reading materials and immigration). 
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To what degree do indicators affect achievement? 



For each indicator, m n ,ans were tabulated showing the degree to which teachers felt indicators 
negatively influenced the academic achievement of their students. The results are ranked in Table 3-5 by 
severity, regardless of how many children were affected. Examining Table 3-5, one can see that urban 
teachers felt that parental substance abuse had the most detrimental impact on students, while rural 
teachers felt that a lack of reading (and educational) materials and low self-esteem most severely affected 
their students. Teachers serving large percentages of Native American children believed low self-esteem 
had the most negative effect on their student population. 

One finding derived from Table 3-5, in conjunction with Table 3-3 and 3-4, deserves particular 
mention-especially in view of the mandates of H.B. 2217 (1988) which directed schools to address parent 
involvement in their programs. As expected, Table 3-4 shows that relatively large percentages of children 
were believed to be affected by a lack of parental involvement. But while Table 3-4 shows that more 
students were affected by lack of parental participation than lack of parental support, Table 5-5 clearly 
indicates that lack of parental support was believed to have a more severe impact on achievement 

Table 3-5 



RANK ORDER OF K-3 AT-RISK INDICATORS BY DEGREE OF NEGATIVE EFFECT 



URBAN 
(n = 1336) 

Sub. abuie by parents 3.41 

Abusive home 3.39 

Few reading materials 3.39 

Low self-esteem 3.33 

Low parent support 3.26 

Emot/beh problems 3 23 

Transience/mobility 3.25 

Recent immigration 3.22 

Poor comm. w/parents 3.20 

Health problems 3.16 

Low par. participation 3.15 

Substandard home 3.15 

Low Eng. proficiency 3.15 

"Latch-key" home 3.12 

Low annual income 3.10 

Sibling dropout(s) 3.U7 

J>2 schools attended 3.07 



Care for Ablings 2.90 
Sub. abuse by child 2.90 
Retained > I time 2.69 



No %4tor($) < 2.50 



RURAL 
(n = 115 8) 

Few reading materials 3.15 

Low self-esteem 3.15 

Poor comm. w/parents 3.10 

Low parent support 3.08 

Emot/beh problems 3 .06 

Low Eng. proficiency 3.05 

Sub. abuse by parents 3.03 

Abusive home IJ.00 



Recent immigration 2.99 

Kvilth problems 2.98 

Low par. participation 2.98 

Transience/mobility 2 . 97 

Substandard home 2.93 

"Latch-key" home 2.93 

Low annual income 2.89 

>. 2 schools attended 2.87 

Care for siblings 2.71 

Retained > 1 time 2.54 



Sibling dropout(s) 2.45 
Sub. abuse by child 2.14 



RESERVATION 
(n = 1124) 

Low self-esteem 3.14 

Sub. abuse by parents 3.13 

Emot/beh problems 3.10 

Few reading materials 3.09 

Abusive home 3.04 

Poor comm. w/parents 3.03 

Low Eng. proficiency 3.01 

Low parent support 3.00 



Low par. participation 2.97 

Health problems 2.96 

"Latch-key" home 2.86 

Low annual income 2.84 

Substandard home 2.82 

Sub. abuse by child 2.75 

Transience/mobility 2 . 74 

Sibling dropout(s) 2.70 

>. 2 schools attended 2.67 

Care for siblings 2.63 

Retained > 1 time 2.59 



Recent immigration 2.21 



TOTAL 
(n = 3618) 

Few reading materials 3.21 

Low self-esteem 3.21 

Sub. abuse by parents 3.18 

Abusive home 3.15 

Emot/beh problems 3.14 

Low parent support 3. 1 1 

Poor comm. w/parents 3.11 

Low Eng. proficiency 3.06 

Transience/mobility 3 .04 

Low par. participation 3.04 

Health problems 3.04 

Recent immigration 3.01 



Substandard home 2.97 

"Latch-key * home 2.96 

T x>w annual income 2.96 

>_ 2 schools attended 2.92 

Care for siblings 2.74 

Sibling dropout(s) 2.72 

Sub. abuse by child 2.76 

Retained > 1 time 2.59 



No indicators) < 2.50 



"32S • ' I ^ j l_j L__^^^__L__ 

?s divide ranges of mean scores and can be interpreted using the following scale: 2.0 = Applies but has NO negative 
eiioti on academic performance; 3.0 = Applies and has SOME negative effect; 4.0 = Applies and has a LARGE negative 
effect 
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Perhaps even more noteworthy is the finding that urban teachers consistently rated indicators as 
having a greater degree of negative effect on their students than did their rural and reservation 
colleagues, despite the fact that the indicators themselves were more prevalent in the other two regions. 
Data reveal that 18 out of 20 indicators were perceived as having a higher degree of negative impact on 
urban students than on rural or reservation students. These findings suggest differences in how educators 
in the three regions perceive the effects of indicators on student achievement. 

Are there patterns of indicators in relation to academic achievement? 

An analysis of the indicators was conducted for students rated in each of five achievement 
categories: very high, high, average, low, and very low. Predictably, this analysis shows that a greater 
number of indicators were perceived as more detrimental for low achievers than for high achievers. This 
holds true for the profiled K-3 students regardless of region (see Appendix A, Table A-2). 

While low achieving children are generally more at risk, there are indicators that effectively 
describe high achieving children considered to be at risk. These indicators vary across regions. For 
example, urban high achieving children were perceived as most affected by parental substance abuse, 
while rural high achievers were most likely to be affected by "latch-key" situations. Reservation high 
achievers were felt to be most negatively affected by health problems. 

A more important question in this analysis was: Are there specific indicators that consistently 
distinguish among students achieving at different levels? To answer this question, indicators were 
examined to see which, if any, had negative effect mean ratings that increased across performance 
categories. Table 3-6 highlights the results of this analysis as broken down by region. 

Table 3-6 



INDICATORS THAT DISTINGUISH VERY LOW FROM VERY HIGH K-3 ACHIEVERS | 


URBAN 


RURAL 


RESERVATION 


No/low English proficiency 
>2 elementary schools attended 
Low parent participition 

Responsible for siblings 


No/low English proficiency 
_>2 elementary schools attended 
Low parent participation 
Retained at least once 


No/to v English proficiency 

Retained at least once 
Responsible for sibling! 


Unique to urban programs: 

Low parent support of child 
Low self-esteem 
Abusive home 
Emotional/behavior problems 

Latch-key situation 
Poor parent communication 


Unique to rural programs: 

Few rcading/cd. n aterials 
Low annual income 
Sibling dropout(s) 
Substance abuse by child 
Substance abuse by parent 
Recent immigration 


Unique to reservation programs: 

Living conditions 
Tr&nflience/mobility 



Table 3-6 reveals that only one indicator consistently appears "predictive" of low academic 
achievement: low English proficiency. But, perhaps more significant is the fact that only five indicators 
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show a trend across performance for reservation children. This implies that most indicators are viewed 
as affecting reservation children equally, regardless of their level of performance. In contrast, ten of 20 
indicators show trends across performance for urban and rural children indicating that these children are 
more readily distinguished as higher or lower achieving children on the basis of specific indicators. 



DESCRIPTION OF THE K-3 PROGRAMS 

This section highlights the specific services and activities undertaken by school districts to serve 
at-risk students. As described in the FY 1989/90 Project Report for the Arizona At-Risk Pilot Project 
(Morrison Institute, 1990), K-3 program services will be discussed in terms of services provided to three 
target groups: students, parents, and staff. 

STUDENT SERVICES 

Improved academic performance, self-esteem, and social development among at-risk children were 
common goals of virtually every district receiving at-risk funding. The specific activities for attaining 
them, however, differed. To find commonalities among them, strategies were analyzed across all 42 
programs. From this analysis, nine broad strategies were identified as most common among the pilot 
sites. These nine formed the organizational framework for collecting student services data from districts. 

Table 3-7 



Strategy # 

1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 



K-3 STUDENT SERVICE STRATEGIES* 



Strategy 



Implement Alternative Delivery Syitemi 
Reduce Student/Staff Ratios 

Alter Claw room Initniction/Implement Curriculum Modification* 

Supplement Individualized Inatniction (Tutoring) 

Provide "Special" Activitiea/Servicei (Support and/or Enrichment) 

Extend Instructional Servicea During the Summer 

Add/Expand Facilitiei 

Enhance On-Going Aaaeeiment 

Add/Expand Counacling/Other Social Service! 



* The numbering of the strategies ii arbitrary. For the take of continuity, the numbering 
lyitem will be preaerved in subsequent discussions ai it wai used by Mormon Institute and 
the diitricti in coding and recording theie activities. 



Implement Alternative Delivery Systems (Strategy H): Supported by at-risk funds, 16 districts 
implemented a total of 23 "alternative" systems for delivering student services. Each differs significantly 
from systems in place prior to H.B. 2217 (1988). Such alternative delivery systems included full-day 
kindergartens, restructured classroom groupings, magnet programs, and other programs specifically 
tailored to the needs of at-risk populations. For example, the Wilson District established "Welcome 
Rooms" for highly transient students who would be enrolled in the district for only a short time. 
"Welcome Rooms" offered more individualized basic skills instruction than regular classrooms, and 
provided transition time for students who might have attended many schools. They also allowed more 
stability in regular classrooms by concentrating student turnover in one central location. 
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Reduce Student/Staff Ratios (Strategy #2): Twenty-two districts sought to increase 
individualized instruction in the "regular" classroom by reducing student/staff ratios through the hiring 
of additional teachers, aides, or both. Instructional aides typically assist teachers by working with 
individual students or small groups to reinforce or provide additional practice on specified skills. 

Alter Classroom Instruction/Implement Curriculum Modifications (Strategy #3): One of the 
most pervasive strategies reported was to adopt curriculum reform and/or different instructional 
techniques in the classroom. Fully 78 percent of the K-3 at-risk programs employed this strategy using 
grant fiinds, while the balance funded such changes through other sources. One subcategory of this 
strategy involves the use of "developmental^ appropriate practices" (DAP) which are research-based 
activities demonstrated to be compatible with early childhood development and learning theories 
(Bredekamp, 1987). Such practices include developing integrated thematic units; teaching listening, 
speaking, reading, and writing as integrated skills through the whole language approach; using 
manipulative objects in mathematics instruction; increasing hands-on activities; and placing students in 
cooperative groups for some instructional activities. 

A second category of instructional change involved increasing the use of computer-assisted 
instruction (CAI) in the regular classroom. Toward this end, seven districts purchased computers for 
specific classroom applications, such as publishing activities that allow students to apply the writing 
process to their assignments. Providing CAI within the classroom was particularly popular in rural and 
reservation programs. 

Supplement Individualized Instruction (Strategy #4): Sixteen districts provided supplemental 
instructional for students through individualized tutorials that met before, during, or after school. In the 
Osborn District, for example, students received reading tutorials in-class as well as after school; in 
Douglas, students at risk of being retained were targeted for tutoring before, during, or after school. 

Provide "Special" Activities/Services (Strategy #5): About half of the districts implemented 
"special 11 activities or services designed to complement or reinforce the regular curriculum. These 
included "instructional support" activities which were highly integrated with the regular curriculum and 
provided systematic follow-up, or "instructional enrichment" activities which were not necessarily 
integrated with the curriculum and did not provide systematic follow-up. In Kayenta, for instance, 
instructional support was prodded in the form of take-home backpacks which contained a book, a tape 
record: of the book, and a cassette player. These could be read, or listened to, by the child and parent 
together. The parent then completed a follow-up survey providing feedback to the teacher about the 
activity. Instructional enrichment, on the other hand, typically included field trips, classroom libraries, 
or supplemental instructional materials. 

Extend Instructional Services During the Summer (Strategy 16): Twenty districts incorporated 
summer school or other summer services into their at-risk programs. While some summer school 
programs emphasized the continuing development of academic skills, others focused on providing students 
with enriching experiences. Many schools opened their libraries to students and families for designated 
periods during the summer. 

Add/Expand Facilities (Strategy #7): Nineteen districts added or expanded facilities other than 
regular classrooms for the purpose of enhancing instructional services. Several of these districts (e.g., 
Wilson and Eloy) established or expanded computer labs. Others added indirect service facilities, such 
as Ganado's closed circuit television service, to all K-3 classrooms. 
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Enhance On-Going Student Assessment (Strategy 18): Jn order to better identify at-risk 
children and their needs, eight districts implemented new or revised screening and assessment procedures 
for students. In addition, several districts revised or modified on-going student assessment procedures in 
keeping with developmentally appropriate practices. In Whiteriver, for example, students were screened 
using Apache language assessments; in Buckeye, a Child Study/Student Assistance Team processed 
referrals from teachers and planned appropriate interventions for specific at-risk students. 

Add/Expand Counseling/Other Social Services (Strategy #9): Eight districts incorporated 
counseling or other types of social services into their programs in order to meet the needs of their at-risk 
students. For example, the Creighton district provided small group counseling and the services of a social 
worker for at-risk students. In Littleton, a community liaison was hired to assist students and their 
families with referral services. 

***** 

Data collected during 1989-90 reflected the strategies used by each pilot site and the number of 
students receiving each type of service. During that first year of the evaluation, districts were directed 
to report a strategy only if it was supported directly by at-risk funds. As the programs evolved, however, 
more an ' more integration occurred between at-risk program strategies and other programs targeting at- 
risk students that were funded by the districts For example, some program sites had full-day kindergarten 
that was supported by at-risk funding, while others offered full-day kindergarten funded from other 
sources. 

Since program integration was an inherent goal of the at-risk grants, Morrison Institute attempted 
to portray the extent to which districts were integrating at-risk funded and district-funded student services. 
In 1990-91, therefore, districts were asked to report all at-risk strategies being employed by their school 
or district, and to indicate whether each strategy was funded by the at-risk grant or by the district (though 
actual student participation numbers were reported only for at-risk funded strategies). As a result, at-risk 
programs could be viewed within the context of other programs already in place in the school or district 
(see Appendix A, Table A-3). 

PARENT SERVICES 

H.B. 22 17 (1988) required districts to identify "procedures for involving parents in the program. " 
An updated analysis of the 42 K-3 at-risk pilot programs revealed four major strategies for involving 
parents. In reporting parent services for 1990-91, districts distinguished at-risk funded activities from 
district funded activities, much as they did for student services. The results are depicted in Appendix A, 
Table A-4. As with student services, this format provides perspective on the range of parent services 
provided, regardless of the funding source. The four major strategies to involve parents are shown on 
the following page in Table 3-8 and subsequently described. 
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Table 3-8 



K-3 PARENT SERVICE STRATEGIES 


Strategy! 


Strategy 


1. 


Increase Home/Community Outreach Efforts 




• Written communication 




a l/ Aa L B 1 AAmmunioattnn I irtolllHino tfclwihflfV* OJlllfl^ 
• VCtDttl vOITllTIUlUCIliaOll lwiw|/J hjiiv wouay 




• Home visits 


2. 


Increase Opportunities for School-Based Involvement 




• Implement formal classroom volunteer programs 




• Include parents as members of school advisory boards 




• Offer school social events 


3. 


Upgrade Parent Skills 




• Offer presentations and workshops 




• Sponsor formal classes (e.g., ESL, G.E.D.) 


4. 


Enhance Counseling/Social Services 



Increase Home/Community Outreach Efforts (Strategy #1): Broadly defined, this strategy for 
involving parents encompassed efforts by school personnel to get information and/or deliver services to 
parents in their homes and communities. Every K-3 program offered some type of home/community 
outreach component, either directly funded through the at-risk program or supported by the district. Such 
efforts included increased written communication and increased community contacts. A number of 
districts also conducted formal home visits, and many aimed to involve parents with their children by 
assigning home instructional activities to be completed jointly. 

Increase Opportunities for School-Based Involvement (Strategy #2): About two-thirds of the K-3 
programs made efforts to increase parent participation through on-site school activities. These efforts 
included three major categories of participation: parents as classroom volunteers, parent membership 
on school advisory boards for the at-risk program, and parent participation in school "events." 

Upgrade Parent Skills (Strategy 13): A majority (86 percent) of districts attempted to involve 
parents by offering opportunities, such as presentations and workshops, for the purpose of improving 
parental skills. Additionally, several districts offered/sponsored formal classes providing parents 
opportunities to upgrade their own skills/abilities, sometimes for college credit. 

Enhance Counseling/Social Services (Strategy #4): About one-half of the at-risk programs 
addressed the physical, social, and economic needs of parents. These services ranged from providing 
referrals to social service agencies that provide family assistance to counseling parents on topics such as 
student absenteeism or drug abuse. 

STAFF SERVICES 

Staff development was a major focus of many at-risk programs during year one of program 
implementation. Although staff development continued to be offered during years two and three, most 
programs began encouraging the implementation and refinement of new teaching strategies rather than 
introducing additional topics. As with student and parent services, some staff services were supported by 
at-risk funds, while others were related to at-risk programs but funded by the district. Primary strategies 
for implementing staff training are depicted in Table 3-9 and subsequently described. 



23 



Table 3-9 



Strategy # 

I. 
2. 
3. 
4. 
5. 



K-3 STAFF DEVELOPMENT STRATEGIES 



Strategy 

Provide Woricahopi/In-service Training 
Encourage attendance at conferences/academies 
Sponsor formal claaaea 

Conduct regularly scheduled at-risk program meetings 
Sponsor school visitations/observationu 



Provide Workshops/In-service (Strategy #1): Virtually all districts provided and/or sponsored 
activities such as formal lectures, training worl shops, and teleconferences. Workshops/in-service training 
addressed a variety of subjects. They were led by peers and professional consultants, and were offered 
on and off-site. During 1990-91, districts tended to reduce the number of regular (e g weekly) 
workshops and, instead, scheduled workshops on an as-needed basis as determined by the teaching staff. 

Encourage Attendance at Conferences/Academies (Strategy #2): Most districts also sponsored 
staff attendance at professional conferences, academies, or other events that offered several days of 
training on a related topic. For example, many teachers attended national or regional conferences 
sponsored by professional organizations such as the Reading Council and the National Indian Education 
Association. 

Sponsor formal classes (Strategy #3): Many districts sponsored classes provided by universities 
community colleges, or professional trainers. Generally, these classes resulted in college and/or district 
credits. At Scales School in Tempe, the at-risk program sponsored a "professor-in-residence" : a nationally 
recognized expert on elementary level at-risk students who provided nine days of on-site consultation to 
the K-3 staff. 

Conduct Regularly Scheduled At-Risk Program Meetings (Strategy #4): In a few districts staff 
development occurred in the context of specially designed program meetings. These meetings typically 
ottered hands-on training in various aspects of program evaluation and implementation, and usually 
incorporated reviews and discussions of professional literature. 

Sponsor School Visitations/Observations (Strategy #5): Several programs sponsored visits by 
their staff to other schools. These visits allowed staff to observe other programs in operation, so they 
might acquire new skills and ideas to enhance their own at-risk prograi is. 

PROGRAM EVALUATION ACTIVITIES AND RESULTS 

Several frameworks exist for describing K-3 programs. The 42 programs have been classified by 
region--urban, rural, and reservation-and by phase. Services are offered in unique combinations of 
strategies and delivery systems, both program-funded and district-sponsored. In sum, K-3 programs are 
diverse and multi-faceted. Within this context, program evaluation efforts needed to be clearly delineated 
Morrison Institute focused on three broad areas of investigation for the purpose of defining "what works" • 
program implementation, program services, and program outcomes. These areas will now be discussed 
along with the evaluation activities and results applicable to each. 
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PROGRAM IMPLEMENTATION 



A major evaluation effort was directed toward determining not only the extent to which programs 
were implemented as planned, but also how they were implemented and how well they were implemented. 
To this end, comprehensive descriptions were compiled documenting such program aspects as planning 
efforts, student identification and placement criteria, staffing patterns (including turnover), and 
communication efforts among and between program staff, other district personnel, parents, and 
community members. As part of this documentation, each district wa. asked to submit participation data 
for students, parents, and staff. All participation data were cross-checked with program descriptions as 
part of a verification process that program services had indeed been implemented as planned. 

Program Participation 

Participation data indicate that 1) all 42 sites offered student services, 2) all attempted to improve 
parent communication and involvement, and 3) all attempted to provide opportunities for staff training 
and professional development. Participation data, aggregated for all programs and averaged for the year, 
are presented in Table 3-10. This table shews that nearly 25,000 kindergarten through third grade 
students were served through one or more services offered by at-risk programs. (Appendix A, Table A-5 
shows how many at-risk and not at-risk students were served by district.) Further review of district 
submitted data shows that a majority of those served (10,703 children) benefitted from the implementation 
of developmental^ appropriate practices in their classrooms, while 305 children were targeted for 
specialized assessment (see Appendix A, Table A-6). 

Table 3-10 also shows the extent of parent services. Written communication efforts reached the 
most parents (9,214) while formal advisory roles for parents involved the least number of parents (126). 
On the basis of district feedback, parents reached via written communication generally were those who 
participated in or were targeted to receive other kinds of services. As a whole, then, one can estimate 
that at least 9,000 individual parents were "reached" by at-risk program efforts; however, the extent of 
their actual "involvement" varies by service (see also Appendix A, Table A-7). 

Regarding staff services, an estimated 5,482 participants attended workshops and other in -service 
activities. This number may reflect duplicated numbers, as staff members attending two or more different 
workshops were counted once for eveiy activity attended. Visiting other schools was *V; least utilized 
staff training strategy (see Appendix A, Table A-8). 
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Table 3-10 



K-3 AT-RISK PROGRAM PARTICIPATION DATA FOR FY 1990-91 


Student Services (unduplicated count) 




24,738 


- At-riik 


18,334 




-- Not At-risk 


6,404 




Parent Services (unduplicated count) 






• Increased home/community outreach effort* 






- written communication 




9,214 


- verbal communication 




3,583 


iormai nomc viiiu 




1,704 


- take-home activitiea 




3,020 


• ocnooi-Dascu invoivcrnciu 






- formal patent classroom volunteers 




60S 


- advisory roles re: at-risk 




126 


- events 




3,565 


• Upgrading parent skills 






— workshops 




2,154 


- formal classes 




691 


• Counseling/social services 




604 


Staff Services (may include duplicated counts*) 






• Workshops/in-service 




5,482 


9 Conferences/Academies 




802 


• Formal classes 




217 


* Formal at-risk program meetings 




1,589 


• Schools visits/observations 




133 


* Duplicated counts are a result of the fact that the same staff member may have participated in two or more different 
kinds of services or activities. 



Quality of Implementation 

While participation data indicate that services were implemented, they do not reveal how well 
services were implemented. The issue of "quality of implementation" was addressed in several ways. 
First, at the onset of the 1990-91 school year, program directors were asked to list the aspects of their 
programs that they felt either contributed to or constituted a barrier to the successful implementation of 
their programs. This list of successful and unsuccessful practices was included in the Arizona At-Risk Pilot 
Project Report for FY 1989-90 and will not be repeated here. The list, however, was used to generate 
items for inclusion on the FY 1990-91 K-3 Teacher Survey. 

Fifteen salient items were selected that were noted to either "make or break" a program. Given 
that these elements were identified by directors at the beginning of the school year, the assumption was 
made that an end-of-year analysis would be useful in determining how well potential barriers were 
overcome during FY 1990-91. During spring 1991, survey respondents were asked to rate each item on 
a 4-point Likert scale. The ratings of these elements are presented in rank order in Table 3-11. 
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Table 3-11 shows that ai! mean scores for the 15 factors were generally positive. Looking at the 
five lowest factors, however, 'key program implementation issues focus on: school and community 
collaboration, communication, the alignment of school and district philosophies toward at-risk students, 
and staff training through both pre-service and in-service. 

Table 3-11 



FACTORS AFFECTING K-3 PROGRA M IMPLEMENTATION (N~ 1021)* 

Very much a contributor to success: 4.0 



Staff commitment to working with at-risk students 3 59 

Availability of funds/resources earmarked for at-risk 3,22 

Staff commitment to working with parents of at-risk student* 3,21 

On-going dialogue/collegiality among teachers on how to assist at-risk students 3. 14 

Administrative support for new programs/change 3,14 

Strong program leadership 3 09 

Program assistance provided by the Arizona Department of Education 3,09 

Evaluation assistance provided by Morrison Institute/site evaluators 2.93 

Teacher "buy-in" for new programs/change 2.93 

Integrated school-district plan for meeting needs of at-risk students 2.92 

Quality of in-service w/respect to at-risk issues 2.89 

Quality of pre-service w/respect to at-risk issues 2.85 

"Alignment' 1 of school-district philosophies toward at-risk students (e.g., testing, curriculum) 2.85 

Clear communication to all staff re: program objectives, implementation, and refinements 2.80 

School and community collaboration in meeting student/parent needs 2.64 



Very much a barrier to success; 1 .0 

Scale: 1.0 = Very much a barrier; 2.0 = somewhat a barrier; 3.0 « somewhat a contributor; 4.0 = very much a 
contributor [NOTE: Teachers were directed to consider characteristics a barrier if they were NOT in place to the extent 
that they "should" be.] 



These results were further analyzed using chi-square tests of significance by two respondent 
variables: region and phase. All chi-square tests were significant. Regional results indicate that urban 
teachers were more satisfied with program implementation than were rural and reservation teachers. 
Moreover, staff at phase I sites were more highly satisfied with program implementation than were staff 
at phase II sites. When phase II district-based programs were separated from phase II school-based 
programs, however, results showed that phase II school-based staff were not only more satisfied with 
program implementation than their district-based colleagues, but also were more satisfied than respondents 
from phase I programs. In other words, program implementation was perceived more positively at sites 
that did not have to "deal" with a district-level bureaucracy. 

Survey respondents were also given the opportunity to comment on program implementation. Of 
1021 respondents, 961 (94 percent) commented on one or more strengths in the areas of: student services 
(51.4 percent responding), parent services (43.0 percent responding), staff services (42.1 percent 
responding), and the program in general (30.0 percent responding). Additionally, changes were 
recommended by 346 respondents (36.0 percent) for student services, 306 (31.8 percent) for parent 
services, 289 (30.0 percent) for staff services, and 217 (22.6 percent) for the "program in general." 
Figure 3-1 depicts this analysis. 
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Figure 3-1 

K-3 PROGRAM STRENGTHS AND RECOMMENDED CHANGES 
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Figure 3-1 indicates that most strengths were noted in student services, whereas most 
recommendations for change were made for the "program in general/ This latter area encompasses issues 
regarding program implementation. Taking a closer look at the "program in general/ Figure 3-2 
illustrates both the strengths and recommendations pertaining to program implementation. A majority of 
program strengths (25.5 percent) were perceived in relation to staff (e.g., their qualifications and/or 
commitment). Leadership and program management were also noted as implementation strengths. 

Figure 3-2 

K-3 PROGRAM IMPLEMENTATION STRENGTHS AND RECOMMENDED CHANGES 
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Of the recommendations made for change, most (26. 1 percent of the total comments in this area) 
reflected a need to improve communication. Additional concerns centered on needs to hire more and 
more qualified staff, improve specific implementation policies/procedures, and provide additional 
resources. It should be noted that under "communication" there were a number of comments reflecting 
teachers' desires to have more input into program planning and implementation. Concerns regarding 
specific implementation policies/procedures were wide-ranging, but encompassed the need for better 
integration and coordination of programs within schools, districts, and communities. 

Interviews with 422 persons (see Appendix A, Table A-9) included discussions regarding program 
implementation. Interview responses regarding overall program implementation revealed that many 
parents and staff had a limited view of the at-risk programs in their districts. One of the few common 
areas of concern, however, involved staff communication and links within schools and districts. Some 
staff said that communication had improved only among teachers, but not between teachers and 
administrators. Still others said that their districts needed better avenues of communication among all 
staff. Yet, several staff interviewed indicated that overall district communication had improved as a result 
of the at-risk programs. Group planning efforts had pulled people together. 

• Said one urban teacher: "We are no longer just a staff of individual teachers... there is 
a feeling of being a group." A reservation teacher added: "There is a team spirit here.... 
There is a pride in improving the school. ..We understand what we have to do." 

Fewer criticisms were aimed at program management than in past interviews. Although 
management issues were still a concern-prompting some staff to call for better administrative support, 
guidance, and direction-staff in some districts indicated that their programs had clarified goals and 
improved leadership during the past year. These factors had helped focus their at-risk programs and 
eliminate confusion and cross-purposes, staff said. Not surprisingly, staff generally perceived that 
effective programs had good leadership. 

Planning issues, however, were still subject to heavy criticism in some districts. Some staff and 
school board members said their districts did not react to problems until threatened with a crisis. Others 
were characterized as having no master plan. 

• Said one exasperated reservation teacher: "It's hard to tell if the program is working 
when it changes every year." 

Regarding program linkages with communities, most positive reports came from urban school 
parents and staff. They said linkages with their communities were improving and new social services were 
being offered on-site at school campuses. Staff from rural districts, however, often expressed dismay that 
their districts were too isolated to establish meaningful community links. 

Several programs were criticized for late implementation and slow progress in effecting changes. 
Also, some staff bridled under forced classroom reforms "from the top" that did not have their input or 
support, even though these same staff admitted that reforms were needed. The notion of systemic reform 
was a common thread throughout the interviews. Although many respondents said that more reform was 
needed within their districts, several staff and parents indicated that great strides had been made already. 

• Said an urban teacher: "Before we got the at-risk funding, the prospects for this school 
making improvement were dismal. This grant is the best boost we could have had. We 
got the training and the materials to implement what we learned." Said a rural teacher: 
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"Prior to having the project, (this school) was a workbook school. There was very little 
communication between grade levels. Now... teachers are working together; 
professionalism has been raised; everyone in K-3 is committed to the success of the 
project." Said a reservation teacher: "Prior to at-risk funding, we had the attitude that, 
'these kids can't fit into the system.' Now we are asking, 'How can we change the 
system to fit the kids?" 

Although, as noted in the interview data, program leadership was not as prominent a theme as 
it had been in the past, a fair number (10 percent) of recommendations for change had to do with 
leadership and program management. At least one hypothesis contends that there is a strong link between 
unstable leadership and poor communication. Anecdotal data indicate that leadership changes often are 
accompanied by changes in philosophy, staff and school organization, and program implementation. Such 
changes were perceived to adversely affect program communication. 

With this in mind, Morrison Institute investigated the extent of administrative staff turnover since 
the inception of each phase I site (i.e., 1988-89). Only phase I sites were analyzed because more 
documentation was available on these sites. Table 3-12 depicts the results of this investigation and shows 
that over twice as many rural and reservation programs have lost their program directors compared to 
urban programs. Moreover, more school and district administrators (e.g., principals, superintendents) 
have left reservation schools than rural. 

Table 3-12 shows that, overall, there has been greater than a 50 percent turnover in key program 
staff and/or administration since the beginning of the programs. High rates of key personnel turnover, 
regardless of their specific affect on communication, are cause for concern-particularly in the more 
isolated and rural programs. 



Table 3-12 





| K-3 PROGRAMS: ADMINISTRATIVE TURNOVER* 


Programs (by Region) 


At-Rbk Troject Director 


Dist/ School Administration 


Urban (8 programs) 
Rural (7 programs) 
Reservation (7 programs) 


2 out of 8 25% 
4 out of 7 57% 
4 out of 7 5796 


4 out of 8 50% 

4 out of 7 57% 

5 out of 7 71% 


TOTAL (22 programs) 


10 out of 22 46% 


13 out of 22 59% 


* Turnover data were extracted from Morrison Institute formative and summative ei 
January 1991; and June 1991, 


/aluation reports: September 1990; 



PROGRAM SERVICES 

Program services encompassed all those services offered to students, parents, and staff as a result 
of the at-risk grant. External evaluators documented and observed those services over time. FY 1990-91 
evaluation efforts focused on surveys and interviews to determine staff and client perceptions of services. 
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A key question in the evaluation of student services was: Do specific strategies (or methods, or 
types of services) appear to "work" better than others? To address this question, K-3 teachers were asked 
to rate specific strategies on a 4-point Likert scale. Teachers were asked whether or not each strategy met 
the academic and/or social/emotional needs of at-risk students (i.e., should be maintained or discarded). 

Recognizing that many K-3 programs integrate services with other existing programs, services 
were evaluated in the total school context, regardless of whether or not the service was an " official M part 
of the program 6 . Teacher ratings of services are presented in Table 3-13. On the whole, Table 3-13 
shows that hiring additional teachers and implementing full-day kindergartens were strategies that worked 
"very well," while counseling services were most "in need of refinement." 

Analyzed by phase and by region, it was found that phase I staff generally rated student services 
higher than did phase II staff, and urban staff rated services higher than did their rural and reservation 
counterparts. And, except for the two most highly rated student services (adding teachers to reduce class 
size and providing full-day kindergarten), regional staff disagreed over what they considered effective 
services. For example, urban teachers rated summer services highly, while rural teachers felt that 
additional materials made a difference, and reservation staff were most enthusiastic about additional aides 
(see Appendix A, Table A-10). 



Table 3-13 



K-3 STUDENT SERVICES EVALUATION (N « 1,021) 




Has worked very well (needs to be maintained as is): 4.0 


Additional teachers) 




3.60 


Full-day kindergarten(s) 




3.58 


Additional aide(s) 




3.41 


Summe service* 




3.40 


Additional instructional material! (e.g., books, backpacks, etc.) 




3.38 


Developmentally appropriate practices 




3.32 


Instructional enrichment activities (e.g., field trips) 




3.32 


Formal individualized/small group tutoring during school 




3.24 


Integrated use of computers in curriculum 




3.22 


Formal before/after school tutoring 




3.18 


Improved facilities other than classrooms (e.g., labs, libraries) 




3.10 


Multi-year classroom(s) 




3.06 


Multi-grade claasroom(s) 




2.95 


Continuous and specialized student assessment techniques (e.g., child study teams) 


2.89 


More counseling/psychological services 




2.64 




Has not worked at all (needs total revision): 1.0 


Scde: 1.0 as Has not worked at all; 2.0 = Has not worked well; 3.0 = Has worked well; 4.0 


= Has worked very well 



6 Even so, an additional analysis was conducted comparing ratings of services in terms of whether or not they were "official" 
aspects of the K-3 program. Pilot districts offering district-sponsored services served as a kind of "non-equivalent control group" 
for pilot districts offering the same service^) under the auspices of their at-risk program. For example, mean ratings were compared 
for those districts offering CAI paid for by district funds versus districts offering CAI a a part of their at-risk program. The analysis 
revealed that, in most cases, survey respondents rate at-risk program services equal to or better than district-sponsored services. 
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Survey respondents were also given the opportunity to comment on student services in the open- 
ended questions. As already depicted in Figure 3-1, the majority of teacher comments 'eg r ing program 
strengths were in the area of student services. Furthermore, more than 1,000 separate comments reflected 
the strengths of student service, while only 424 comments suggested changes. Figure 3-3 represents the 
breakdown of student service strengths and changes as based on the qualitative analysis of all comments. 
When possible, comments were categorized around the nine general strategies discussed earlier in this 
chapter. 

Figure 3-3 shows that the majority of perceived strengths were related to altered Jassroom 
instruction and/or curriculum modifications (strategy #3). Many comments reflected positively on efforts 
to incorporate the use of developmentally appropriate practices in classrooms, or applauded specific 
techniques such as "Math Their Way" and whole-language approaches to instruction. Additional perceived 
strengths included: district-specific initiatives and programs that may or may not be "official" at-risk 
program services (e.g., the Welcome Room; ESL program; Chapter I); reduced student-staff ratios 
(strategy #2, which includes the addition of teachers and aides); strengthened efforts to individualize 
instruction (strategy #4, which encompasses tutorial programs); and full-day kindergarten programs 
(included within strategy #1). 

Figure 3-3 aiso shows teacher recommendations for changes in student services. Many comments 
(23.1 percent of the total) noted general concerns and provided recommendations such as: "maintain 
services as is," "need more services," and "work with more students." Other recommended changes 
focused on the need to further reduce student-staff ratios, improve/expand the implementation of 
developmentally appropriate practices, and/or modify the curriculum. Miscellaneous comments (i.e., 
"Other") included recommendations such as "give away cafeteria leftovers". 



32 



9 

ERIC 



41 



Figure 3-3 

K-3 STUDENT SERVICES STRENGTHS AND RECOMMENDED CHA1 ."GES 
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Student services were also the subject of interviews from which several broad themes emerged 
regardless of region or focus group. First, respondents praised those school districts that had hired 
additional teachers and aides in an effort to reduce class sizes and student/staff ratios. The main benefit 
of these efforts, parents and st f said, was that students received extra individual attention. 

• Regarding one student who had benefitted from extra attention, an urban teacher said: 
"(Before) they would have just dumped him in a special education class. Nothing was 
really wrong with him. He just needed time to develop." 

Along the same lines, parents and staff often applauded tutorial programs whether they occurred 
before, during, or after school. 

• Said an urban teacher: "I don't know how we functioned without (the tutors)... it is very 
difficult to individualize to the extent that some children need, and the in-class tutoring 
program has made this possible." A reservation teacher echoed these sentiments: "We 
might be sending non-readers on to third grade if it were not for this one-on-one 
opportunity." A reservation mother said simply: "My son feels that he is being helped." 

Several respondents, however, expressed specific concerns about tutorials. Some feared that after- 
school tutorials made the school day too long for young learners. Others said that, while they approved 
of in-school tutorials, in practice the sessions were too short or too infrequent to sustain change. 

• Said one reservation teacher: "You can't expect such a brief time to make an impact." 
Another teacher added: "[A child] needs more than 20 minutes [two days a week]." 

Regarding developmentally appropriate practices (DAP) incorporated into the curricula of many 
schools, parents and staff generally agreed that they worked. These practices, said parents and staff, had 
made language arts and math "fun" for children. They had also helped raise students' self-esteem and 
enthusiasm for school. Among the comments: 

• "The holistic curriculum is why my son is in this school," said one urban mother. "The 
(at-risk program) sneaks learning in with the fun stuff," a rural parent said. "(My 
daughter) is excited about school," said a reservation father. 

But many staff pointed out that a transition to developmentally appropriate practices requires a wealth of 
new instructional materials. In many cases program funds were allocated to purchase these new materials, 
prompting one veteran reservation teacher to call it "a dream come true." Nevertheless, staff and parents 
said that many more new educational materials were needed, especially in rural and reservation schools. 

• Lamented one rural teacher: "You can't use whole language without books." A 
frustrated reservation teacher asked: "How can we do hands-on activities when we don't 
have materials for the kids?" 

A few variations in responses did occur by region. Urban parents and staff, for example, 
frequently cited full-day kindergarten as on*~ of the most effective components of an at-risk program with 
one parent noting it was an "absolute necessity." Rural and reservation respondents, on the other hand, 
more often praised the effectiveness of bilingual stojQT-and cited the need to hire more. Reservation staff 
made the same comments regarding K-3 counselors. The counselors were effective, they said, but more 
were needed. 
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Parent services were examined mach as were student services. Using a 4-point Liken scale, K-3 
teachers rated whether or not specific strategies "worked for involving parents of at -risk students in their 
child's education/ As was done for student services, parent services were evaluated in the total school 
context, regardless of whether or not the service was an "official" part of the program. Teacher ratings 
of services are presented in Table 3 14. On the whole, Table 3-14 shows that teachers rated social events 
as working better than other parent involvement strategies. The ratings of most other strategies cluster 
together. More counseling/psychological services, however, were again rated as most in need of 
refinement (cf. student services, Table 3-13). Classroom parent volunteer programs were the second 
lowest rated strategy. 

Table 3-14 



K-3 PARENT SERVICES EVALUATION (N « 1,021) 

Has worked very well (needs to be maintained as is): 4.0 



School social events 3.20 

More verbal communication between the parent and teacher/school 3.09 

Parent workshops 3.05 

Structured classes for parents (e.g., GED, ESL) 3.03 

Formal home visits 3.02 

Take-home activities for parent* to work in their homes with their children 2.97 

More written material mailed/sent to the home 2.95 

Formal advisory n>!r* for parents 2.88 

Classroom parent volunteer program(s) 2.69 

More counseling/psychological services 2.66 



lias not worked at all (needs total revision): 1.0 
Scale: 1.0 = Haa not worked at all; 2.0 = Haa not worked well; 3.0 = Has worked well; 4.0 = Hat worked very well 



Analyzed by phase and by region, the data show that phase I district personnel rated parent 
services higher than did phase II district staff, while rural program staff rated parent services higher than 
did their urban and reservation counterparts (see Appendix A, Table A-10). All regions shared the 
perception that school social events were effective in enticing parents to school. Also, urban and rural 
staff both felt that increased efforts to communicate with parents in person, (i.e., verbally) had been 
effective. Rural staff, more than their colleague?, also supported educating parents by offering classes 
(e.g., English as a Second Language; G.E.D. classes). 

Over twice as many open-ended survey responses addressed parent service strengths than 
suggested changes (671 comments regarding strengths; 327 comments regarding changes). A majority 
of positive comments pertained to parent workshops and/or formal classes which survey respondents felt 
were successful in "reaching" parents (especially hands-on, or "make-and-take," workshops). 
Additionally, respondents said that increased communication efforts held promise for involving more 
parents of at-risk youngsters. In particular, person-to-person encounters were cited as more effective than 
mai'ed or written communication efforts. 

Most recommended changes reflected a general desire for improvement (e.g., need to reach more 
parents; parents need to be more involved). Many comments mentioned a continuing need to reach 
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parents through contacts, workshops, and formal classes. Results of the open-ended question analysis 
regarding parent services strengths and changes are portrayed in Figure 3-4. 

Figure 3-4 

K-3 PARENT SERVICES STRENGTHS AND RECOMMENDED CHANGES 



Strengths 
(671 comments) 




Home School- Workshops/ Social Specific General 
Outreach Based Classes Services 



Recommended Changes 
(327 comments) 




Home School- Workshops/ Social Specjf|c Geneni| 0tW 
Outreach Based Classes Services 



Specific (e.g., local district procedures) 
General (e.g.. like:" "dislike") 
Other = miscellaneous 
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Interviews with parents and staff indicated agreement on many issues pertaining to parent services 
regardless of region. One of the most frequently made comments was that home visits by school 
personnel had increased as a result of at-risk programs. New "parent facilitators" were especially credited 
with improving school/home contacts, particularly at urban and rural schools. 

• Said one rural parent: "My kids have gone to a lot of schools. There never was any 
reason to get involved. [But here, the parent facilitator] went from door to door; she kept 
coming to our house and saying: 'If you care, please come. This will help our kids. '...So 
now I volunteer." 

Parents and staff also agreed that other school contacts had improved as well. These included 
notes sent home, telephone calls from teachers, and informal visits made to the school by parents. 
Nevertheless, many more such contacts were needed, respondents said, and some changes were needed. 

• Said one urban staff member: "You can't send out notices and expect people to come." 
A rural parent suggested: "The school should try to work more directly with parents 
rather than using the children as middlemen." 

Staff particularly felt that efforts were needed to concentrate on those parents who never had 
contact with the schools. 

• Observed one urban staff member: "It is still extremely difficult to reach transient 
parents. They don't have the time and they don't have the energy." 

Regarding the results of parent training efforts, parents and staff tended to agree that certain 
kinds of workshops had worked well. Singled out for praise were "make'n'take" workshops related to 
educational activities. Some parents, however, suggested more of these workshops be held in their native 
languages. Special adult GED and ESL classes offered through the schools also received favorable 
comments. 

• Said one rural teacher: "Most parents starting out in the class have only a second or 
third grade education. Now they are reading in Spanish and English... These parents are 
excited about reading with their children." 

A rural parent, speaking with the assistance of an interpreter, added: "I want to learn to speak English 
because my children are speaking better than me/ 

Parents and staff repeatedly specified two effective strategies for bringing parents to the schools. 
The first was to invite parents to special events, such as science fairs, which focused on student work. 
The reason this strategy worked, they said, was simple. 

• Explained one reservation teacher: "Kids' school work is what we all have in 
common." 

The second strategy was to ask parents to help chaperon field trips. 

• One rural parent related how it worked for her: "At first, I went on a field trip. Then 
I was relaxed around the teachers, so I decided I wanted to help out in the classroom." 
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Reservation parents and staff also singled out take-home activities as effective parent involvement 
tools. Of particular value were home-based shared reading activities. 

• One teacher said: "It's a good form of communication with parents/ Another teacher 
quoted one of her students: "Dad just loves these books! 11 

Yet, although parent involvement may be increasing in general, staff said that the parents teachers 
needed to see the most are the hardest to reach. 

• A reservation teacher commented: H [I don't] see parents of at-risk kids involved at the 
school." 

Parents and staff cited several barriers to parent involvement. Among those were: language 
barriers (urban parents), transportation difficulties (urban and reservation parents), lack of importance 
placed on education (urban, rural, and reservation parents), and lack of time (urban parents). 

• Regarding the latter, one urban administrator said: "It is difficult for parents to be 
involved in school wnen they are spending their energy trying to survive." 

Several parents commented that a "welcoming" atmosphere was essential to encourage parent 
involvement. Many said their schools were now trying to make them feel more comfortable. 

• Said one reservation parent who had earlier had two children retained by her school: 
"Years ago I felt like I didn't belong here at the school. [Now] I do belong here." 

But not all parents agreed on this issue. 

• Although a parent at one school in an urban district said, "I have never felt so wanted," 
parents at another school in the same district commented bitterly: "Some of the teachers 
don't want us here. Teachers don't think they can learn anything from parents." 

Staff Services 

Evaluation efforts were directed toward determining staff satisfaction with development and 
training. Many of the original K-3 program proposals stated that increasing student skills would be 
accomplished by training staff to work more effectively with students. In lieu of being able to directly 
determine the "impact" of staff training on student achievement, survey respondents were asked to reflect 
on what training activities helped staff "to work effectively with at-risk students." Table 3-15 presents 
the results of this evaluation. It shows that all staff training activities included on the survey were rated 
above a 3.0, indicating that all were perceived as at least "working well." ADE conferences/academies 
received the highest rating, while regularly scheduled in-services received the lowest. 
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Table 3-15 



K-3 STAFF SERVICES EVALUATION (N ■ 1,021) 

Has worked very well (needs to be maintained as is): 4.0 



ADE i porno red K-3 conference and/or Academic! 3.34 

Other conferences/academies 3.30 

Formal instruction through a college course or other training class 3.29 

Workshops/In-service delivered by outside consultant/trainer 3.21 

Regularly scheduled program planning/development meetings (i.e., specific to at-risk) 3*20 

Workshops/In-service delivered by district staff 3.16 

School/program visits and/or observations outside of own school 3. 13 

Workshops/In-service provided on a regularly scheduled basis 3.1 1 



lias not worked at all (needs total revision): 1.0 
Scale: 1.0 = Has not worked at all; 2.0 = Has not worked well; 3.0 = Has worked well; 4.0 = Has worked very well 



Analyzed by phase and by region, results for staff services parallel those found for student 
services. Phase I staff rated more services as working very well than did phase II district personnel, and 
urban staff rated services higher than did their rural and reservation colleagues. Notably, both urban and 
rural staff rated formal training opportunities (e.g., college classes) most effective, while these were rated 
lower by reservation staff (who had less access to them). Appendix A, Table A-10 depicts the rank 
ordering of staff services by region. 

Of the 419 open-ended comments regarding staff service strengths, most addressed the general 
nature of training opportunities such as workshops, conferences, and formal classes. Additionally, 
respondents felt that regularly scheduled planning meetings were helpful when they provided opportunities 
for staff to share ideas. Also applauded were specific district policies/procedures, such as providing 
incentives for attending training opportunities (see Figure 3-5). 

Conversely, the majority of changes recommended (37.2 percent of the 247 comments) 
emphasized the H general H need for additional staff training. Specific recommendations encompassed a 
diverse range of suggestions including the need for follow-up on in-service, mandatory attendance, non- 
mandatory attendance, mentoring, better teacher evaluation systems, additional release time, and other 
incentives to participate in staff development. In the category of "workshops," the need for three general 
kinds of workshops was mentioned repeatedly: 1) orientation workshops regarding at-risk students and 
programs; 2) workshops for teachers on how to plan and implement parent involvement programs; and 
3) workshops for aides and community members (see Figure 3-5). 
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Figure 3-5 

K-3 STAFF SERVICES STRENGTHS AND RECOMMENDED CHANGES 
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Interview data regarding staff development issues showed few major discrepancies when analyzed 
across regions. Instead, most perceptual differences in the data could be "iributed to variations among 
school districts. Some placed heavy emphasis on staff development; others did not. 

Many staff praised their district's increase in staff development opportunities. Workshops and in- 
services, they said, provided many new ideas and made them more sensitive to the needs of at-risk 
students, 

• Said one urban teacher: "What we were doing before was not working. Failure was 
built in because we weren't addressing the problems of unprepared kids." Another 
said: "Now we're giving (kids) the freedom to learn from each other and their 
environment, and they are soaking it up." 

The activities that were rated most highly were those in-services that were regular, frequent, and 
concentrated on implementing new teaching strategies, not just "talking theory. " Teacher-led workshops 
were rated especially high in terms of relevancy and applicability to the classroom. 

Training was criticized that was not pertinent to the student population, had no follow-up, or 
didn't promote classroom application. Also, in some districts, teachers complained that they had been hit 
with "too much too fast." They said they had been "saturated" with new ideas, yet had been given no 
time to implement these ideas as classroom strategies. These teachers suggested their districts concentrate 
on one new strategy at a time, then follow that strategy through to implementation. 

Teachers and aides agreed that aides needed more training to develop tutoring skills and 
understand new curriculum. Unfortunately, they said, aides were often left out of staff development 
activities. Staff also agreed that some teachers and aides needed training on how to work together 
effectively. Some aides, they said, were not always used appropriately in the classrooms. Instead of 
actively tutoring at-risk students, one reservation aide said, they "just do bulletin boards." 

At schools where staff development was not a priority, staff often called for more workshops and 
in-services. Rural and reservation staff particularly favored more workshops held on-site as opposed to 
holding them in distant cities. These same staff also suggested more observations of other school 
programs to garner new ideas. 

• As one rural teacher commented: "I don't know what to do. We need new ideas for 
how to work with at-risk. We need resources." 

Many staff said they had the opportunity to travel to off-campus conferences. These conferences 
were ofter, praised, but some schools were criticized for not taking advantage of the opportunity to follow 
up on the experience. 

• Said staff at ev e urban school: "There is a lot of time spent attending conferences, but 
there is not time to share what I've learned." 

Another frequent topic of concern for staff was the relationship between school planning and 
intra-staff communication. Staff said that regular g r ade-level meetings, cross-grade team meetings, and 
scheduled planning time with other teachers promoted staff unity and provided invaluable avenues for 
communication and sharing. 
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• Said one reservation teacher about the hidden benefit of a staff meeting convened to 
discuss selection of classroom materials: "It made us think in bigger terms about the 
curriculum." 

In some districts, staff ca 'ed for more resource teachers and K-3 specialists to assist teachers 
Schools that employed such resource specialists often said communication among staff had improved as 
a result. In contrast, schools that did not provide either resource teachers or planning opportunities fo r 
their teachers were often characterized as having poor lines of communication. 

PROGRAM OUTCOMES 

K-3 at-risk outcomes were examined in several ways. First, pilot site personnel were asked their 
perceptions regarding program outcomes. Second, student outcome data-including achievement test 
scores, absenteeism, and promotion/retention rates-were collected via the cohort study and retention rate 
survey. Third, efforts were made to collect site-specific outcome data from participating districts through 
self-reports. 

Perceptions of Program Outcomes 

Seven program outcomes 7 were included as items to be rated on the K-3 Teacher Survey. Two 
outcomes pertained to student outcomes, cne to parent involvement, one to staff development, and three 
related to "overall program" outcomes. Table 3-16 shows that the mean score of all items was below the 
score indicating "achieved to a high degree." Two outcomes-improved parent involvement for parents 
of at-risk students and improving linkages with community-based organizations-were most negatively 
perceived (falling below a mean score of 2.5). 

Table 3*16 





K-3 PROGRAM OUTCOMES EVALUATION (N « 


1,021) 


Type of Outcome 


Outcome 


Achieved to a very high degree: 
4.0 


Student 
Student 
Staff 

Program in General 
Program in General 

Parent 
Program in General 


Improved academic achievement of at-rilk ftudenta 
Improved student self -esteem of a' risk students 
Increased staff skills for working w/at-riak students 
Better linkages among school programs 
A comprehensive educational program 
Improved parent involvement for parents of at-risk 
Better linkages with community-based organizations 


2.97 
2.74 
2.65 
2.62 
2.58 
2.46 
2.30 

L0: 

Achieved to a very low degree 


Scale: 1.0 = Very low degree; 2.0 = Low degree; 3.0 = High degree; 4.0 = Vciy high degree 



7 These seven program outcomes were leaned from reviews of pilot program proposals and represent commonly shared or 
globally desirable outcomes for all programs. 
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These data, analyzed by phase and by region, indicate that phase I staff rated more outcomes as 
having been achieved to a very high degree than did phase II staff. In accordance with the findings for 
student and staff services, urban staff rated outcomes higher than did their rural and reservation 
colleagues. Staff in all three regions, however, rated the student outcomes higher than other outcomes. 

Survey respondents also were asked specifically to reflect on the "bottom line" outcome for the 
at-risk programs as specified by H.B. 2217 (1988), i.e., "...that at-risk pupils exiting grade three meet 
the minimum competency requirements prescribed by the state board." They were asked: "Do you feel 
that your district/school efforts to help at-risk pupils are, indeed, keeping students "on-track" such that 
the students will exit third grade having mastered competency requirements?" Table 3-17 presents teacher 
responses by region and phase. 

Table M7 



ARE K-3 STUP 


"NTS "ON-TRACK?" (N - 986) 


REGION 


Urban 


Rural 


Reservation 


TOTAL 


PHASE 


I 

(n - 257) 


II 

(n = 156) 


I 

(n = 139) 


II 

(n = 134) 


I 

(n = 235) 


II 

(n = 65) 


YES, to a large 
degree 


38% 


21% 


23% 


22% 


30% 


28% 


28% 


YES, to tome degree 


54% 


68% 


63% 


71% 


5t% 


71% 


62% 


NO 


7% 


10% 


15% 


6% 


11% 


0% 


10% 


No resporue 


<1% 


1% 


<1% 


1% 


1% 


;% 


<1% 



Table 3-17 shows that a majority of respondents said they do believe that they are keeping 
students "on track;" however, there is a diversity of opinion as to the degree to which this is happening. 
Specific responses reflected this diversity: 

• YES, to a large degree: 

"I feel that the students will have a greater language vocabulary-able to communicate more 
because of the experiences that have become available through summer school, enrichment, take- 
home packages, and the visiting artists program. Parents are more involved with their child's 
learning.../ 

"The teachers in our school are using a variety of resources and measures to see that students 
reach their potential." 

• YES, to some degree: 

"The district is only beginning to organize a consistent barometer for assessment which 
determines mastery and promotion." 

"I feel that in most cases our school/district efforts are keeping kids on track. However, the level 
at which some at-risk pupils enter our classrooms is so far below grade level...." 
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• NO: 

"I feel that many third graders will go on to fourth grade with too low of a reading level...." 

"Academic achievements for students have not been adequately supported." 

Note that according to Table 3-17, nearly four out of every 10 staff members in urban phase I 
districts said they felt strongly that they were are keeping students on back, while in urban phase II 
districts, only two out of 10 said the same. Aggregated by region, the data show that urban and 
reservation staff were generally more positive that they were keeping pupils on-track (31 percent and 29 
percent, respectively, strong "YES" responses) than were rural respondents (21 percent strong "YES"). 

Certainly, a primary goal of the at-risk programs is to keep children "on track;" yet some may 
interpret this to mean advancing students from grade to grade regardless of their skill levels. With this 
in mind, Morrison Institute staff asked survey respondents the following question: "Many districts are 
adopting policies which prevent or discourage students from being retained. This is giving rise to some 
concern over children being promoted to a higher grade without appropriate skills. Do you feel children 
are being "socially promoted" in your district/school?" Table 3-18 summarizes the responses to this 
question by region and by phase. 



Table 3-18 



"ARE K-3 STUDENTS BE 


ING SOCIALLY PROMOTED?" (N - 986) 


REGION 


Urban 


Rural 


Reservation 


TOTAL 


PHASE 


I 

(n «= 257) 


II 

(n = 156) 


I 

(n = 139) 


II 

'« » 134) 


I 

(n « 235) 


II 

(n ■ 65) 


YES, to a large 
degree 


15% 


25% 


16% 


13% 


18% 


14% 


17% 


YES, to some degree 


51% 


49% 


54% 


68% 


54% 


62% 


55% 


NO 


34% 


24% 


30% 


18% 


26% 


20% 


26% 


No reaponae 


<1% 


2% 


0% 


1% 


2% 


4% 


2% 



Looking at Table 3-18, it is apparent that a majority of respondents in all districts felt that some 
degree of social promotion exists in their schools/districts. Responses from urban phase I districts were 
most positive that social promotion was not an issue (34 percent "No" responses), while responses from 
rural phase II districts were least positive (only 18 percent "No" responses). When data are aggregated 
by region, the results show that 30 percent of urban respondents and 24 percent of both the rural and 
reservation respondents felt they were not socially promoting students. Comments typifying teachers' 
opinions were as follows: 

• YES, to a large degree: 

"Students are being promoted and they do not have the skills necessary to perform at the next 
level." 

"Too many children are being promoted for age-grade reasons, but there is no way they can do 
the work for their age-appropriate classroom." 
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• YES, to some degree: 



"Students who have already been retained even in kindergarten are not allowed to be retained 
again. I consider this to be 'social promotion/" 

"Many parents refuse to let their children be held back." 

• NO: 

"We have many remedial programs that help out at-risk students." 

"My district does not 'socially promote' students. Each child is carefully evaluated to determine 
the best placement for the following year. Many factors are considered in this evaluation." 

In summary, the K-3 Teacher Survey collected respondents' perceptions of program outcomes. 
Results (Table 3-16) indicate a perception that student outcomes have been achieved, if only moderately. 
When this result was examined in light of the "bottom line" of these programs-to keep kids on track in 
mastering skills, survey respondents said they felt they were keeping students on track, at least to some 
degree (Table 3-17). At the same time, there was substantial concern that children might be advanced to 
higher grades without appropriate skills (Table 3-18). 

Student Mobility: Can programs make a difference if the students don't remain in school? 

Teachers expressed one additional concern regarding program and student outcomes: no matter 
how effective programs were, they could not make a difference if students did not stay in school. Given 
this concern, Morrison Institute staff investigated mobility within the context of the multi-year data 
collection effort initiated in FY 1989-90. 

An original "cohort" of 3,958 K-3 students was identified in FY 1989-90, and data were retrieved 
to the degree possible for FY 1988-89. During FY 1990-91, districts continued to track students in this 
cohort group. Of the original 3,958 students, 251 students (or 6.3 percent) were "lost" between sporting 
years (i.e., records were missing or unable to be "matched" for FY 1989/90 and FY 90/91 \ ^erefore, 
complete or partial data were reported for 3,707 students for FY 1990-91. Of these student, 670 either 
did not reenroll (n=659) or data were not recorded (n= 1 1). Thus, 18.1 percent (n=670) of the students 
in the original cohort were lost for the study. 

Of the 3,037 children who did reenroll, an additional 223 students were lost during the school 
year, leaving 2,814 pupils from the original cohort (71.1 percent). In sum, nearly three out of every 10 
students (28.9 percent) were unable to be tracked longitudinally. The greatest attrition occurred between 
school years, with many children never reenrolling in school. However, school-year "transience" 
accounted for additional turnover. 

Student attrition, or mobility, was also examined by phase and by region. Phase I and II 
district/school data were virtually identical. Both types of sites lost roughly 18 percent of their students 
between school years and an additional 8 percent of their students during the FY 1990-91 school year. 
Regional differences, however, were identified and are presented in Table 3-19. 

Table 3-19 indicates that urban/suburban district summer attrition rates were highest of the three 
regions, although rural district rates were only slightly less. School-year attrition rates for urban and rural 
districts were also roughly equivalent. Total attrition rates (i.e., all students lost from the FY 1990-91 
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cohort) for urban and rural districts showed that both lost over one-quarter of their student population 
in a 12-month period. Reservation districts had more stable student populations. 

Table 3-19 



K-3 STUDENT AT 


TRTTION RATES 


REGION 


1990-91 
Cohort 


Summer Attrition 
I % of cohort 


Reenrolled 
(1990-91) 


School-year 
attrition 

if % reenrolled 


Total 

ATTRITION 
# % of cohort 


Urban/Suburban 


1,370 


302 22.0 


1,068 


87 8.9 


389 28.4 


Rut, 


1,368 


262 19.2 


1,106 


103 10.3 


365 26.7 


Reservation 


969 


106 11.0 


863 


33 4.0 


139 14.3 


TOTAL 


3,707 


670 18.1 


3,037 


223 7.3 


893 24.1 



Individual district data revealed that, for many of them, student attrition rates were even higher 
than those reported in Table 3-19. Half of the urban districts/schools for which data were longitudinally 
available (6 of 12 sites) showed attrition rates over 25 percent-two of these were over 50 percent (urban 
rates ranged from 0 percent to 53 percent). Eight of the 15 rural districts/schools had attrition r ites over 
25 percent-one over 50 percent (rural rates ranged from a low of 16 percent to a high of 52 percent). 
Of the 11 reservation sites for which data were available, only one distri i "lost" 25 percent or more of 
their students (reservation rates ranged from a low of 4 percent to a high of 48 percent). 

Interview and anecdotal data suggest several reaso.is for high percentages of student attrition. In 
urban areas, many families moved from one urban district to another due to unemployment or "to keep 
one step ihead of the law/ Among rural populations, migrant labor accounted for some proportion of 
student mobility. Other contributing causes of mobility and transience, however, remain to be 
investigated. What is clear is that teacher concerns regarding student mobility generally appear to be 
valid. 

Student Outcomes 

In order to gauge whether or not programs made a difference in students' academic performance, 
two primary student outcomes-attendance and achievement-were examined for those K-3 cohort students 
for whom three years of data were available. A separate study was conducted to investigate student 
retention among pilot sites. 

K-3 Student Attendance: Using the hypothesis that student learning is adversely affected by high 
rates of absenteeism, evaluators examined variations in student attendance during the program 
implementation period. In the Arizona At-Risk Pilot Project FY 1989-90 Project Report (Bierlein et al., 
1990), a two-year trend was reported that reflected significant decreases in absenteeism among pilot 
program students. This trend continues to be observed in the third year of the project, 
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As noted in 
end of the FY 
children (n's vary 




ious section, 2,814 students from the original cohort group remained at the 
"ool year. Absenteeism rates' across three years for roughly 2,353 of these 
by year) are depicted in Figure 3-6. 



Figure 6 shows absentee rates for the total cohort group and the subgroups of students comprising 
the cohort. These subgroups are children who have been tracked since FY 1988-89 with one exception: 
kindergartner students identified in 1989-90 have only been tracked two years (through their 1990-91 first 
grade year). 

Figure 3-6 



ABSENTEE RATES FOR K-3 COHORT STUDENTS (N » 2,35.0 



% of time absent 




K-Sacond 




FVw-ThH 



S*cond-Fcwrt MukHrwk 



TOTAL 



D FY I9B8-89 ■ FY 1989-90 □ FY 1990-91 



Figure 3-6 shows that absenteeism rates have declined steadily throughout the implementation 
period of the at-risk programs for the total group profiled. Rates for subgroups of children within the 
cohort (e.g., kindergarten students in FY 1988-89 tracked through second grade) do tend to vary, 
however. 

Data were analyzed by region and phase using subgroups of children representing each region 
and phase. Table 3-20 summarizes differences in absenteeism by region and phase, and shows that 
reservation students consistently had the highest absentee rates. Rural programs showed the most 
consistent trend in decreasing absenteeism. In addition, phase I programs consistently had less 
absenteeism than phase II districts/schools. 



' These arc calculated by taking the average number of days absent divided by the average number of days enrolled. 
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Table 3-20 



K-3 ABSENTEE RATES BY REGION AND PHASE 




FY 198849 


FY 1989-90 


FY 1990-91 


REGIONAL BREAKDOWN 








0 Urban/Suburban 


4.4 


4.3 


5.6 


• Rural 


5.2 


4.9 


4.7 


* Reservation 


6.0 


5.2 


5.9 


PHASE BREAKDOWN 








• Phase I 


5.1 


4.3 


4.6 


• Phase II 


5.9 


5.5 


6.1 



Reduced absenteeism implies that children received M additional* instructional days. Districts also 
"increased" instructional time through such strategies as full-day kindergarten, one-on-one tutorial 
assistance, and summer school. Thus, it may be concluded that both the quantity and quality of 
instructional time were addressed by at-risk programs. 

K-3 Student Achievement: ITBS scores for approximately 550 students for whom three years 
of data were available were examined to discern any changes in test performance over time. Changes in 
student performance cannot be attributed solely to participation in an at-risk program, nevertheless, three- 
year trends in student achievement were examined using normal curve equivalent (or NCE) scores. The 
trends found are particularly descriptive when presented by region. Thus, Figure 3-7 on page 49 portrays 
NCE scores for reading, language, and math subtests of the ITBS, by reg ; )n and by total. 

Figure 3-7 shows that net NCE gains were made in reading and language over the three-year 
period, while math NCE scores declined. (NCE scores compare a student's gains or losses with the 
average gains/losses for all students who took the test.) In other words, reading and language subtests 
revealed average growth (or more), while the math subtest showed less than average growth. Further 
inspection of Figure 3-7 shows that aggregate trends can be misleading. For example, between 1990 and 
1991, pupils in all regions showed gains in reading, but actually only urban children made steady growth. 
Also, most of the overall decline in math scores can be attributed to reservation children who represented 
nearly half of the children profiled. 

One thing is important to remember: declines in NCE scores do not mean that students made no 
progress. In fact, looking at grade equivalent (G.E.) scores, one can see that all students, in all areas, 
showed steady developmental progress ov< r the three year period. As shown in Figure 3-8 on page 50, 
grade equivalent scores (shown along the left axis) are contrasted with NCE scores (shown along the right 
axis). Scores are presented for four subgroups of students tracked between FY 1989 through 1991: 

• K 1 = 1990 kindergarten followed through 1991 first grade (n = 10), 

• K 1 2 = 1989 kindergarten followed through 1991 second grade (n = 6), 

• 1 2 3 = 1989 first grade followed through 1991 third grade (n = 184), and 

• 2 3 4 =^ 1989 second grade followed through 1991 fourth grade (n ~ 318 s ". 



9 G.E.t are reported only for those children whose NCE scores could be meaningfully interpret with specific reference to 
grade level tests and scores; therefore, Figure 3-8 excludes children reported in multi-grade classrx ns and does not reflect the 
"TOTAL H population. 
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Figure 3-7 

AVERAGE ITBS NORM REFERENCED TEST SCORES (K-3 COHORT <. 556*) 



Reading 



NCE 




Urban (n»l02) Kuril (n-197) R«»rviUon (n-257) TOTAL (N-556) 



Language 



NCE 




Urban (n«IOI) Rural (n-196) R««rv»Uon (n«2SS) TOTAL (N-553) 



Math 



NCE 




Urban (n-102) Rural (n«l9S) Reservation (n«2S6) TOTAL (N-553) 



* n's vary slight y year and by subteat □ FY 1988-89 □ FY 1989-90 □ FY 1990-91 
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Figure 3-8 
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One additional observation can be made about the ITBS data presented in Figure 3-8. Although 
G.E. scores showed consistent developmental progress, it should be noted that a child exiting third grade 
at mid-grade level (e.g., 3.5) would still be perceived as entering fourth grade below grade level. Thus, 
teacher concerns about grade level promotion of at-risk students are supported. 

A final analysis of ITBS scores revisits the legislative mandate that " at-risk pupils exiting grade 
three meet the minimum competency requirements prescribed by the state board. " Two consecutive sets 
of ITBS NCE scores (FY 1989-90 and FY 1990-91) for third grade students wtre compared in this 
analysis. The hypothesis was that gains in third grade students' scores across consecutive years might 
reflect changes in the school (e.g., the implementation of at-risk programs). Again, changes cannot be 
strictly attributed to the implementation of at-risk programs because there are too many other variables 
which confound the analysis. Nevertheless, Figure 3-9 shows that exiting third grade students in FY 
1990-91 were performing higher than their FY 1989-90 counterparts in reading, language, and the 
composite. Math NCE scores, however, declined. 

Although not represented in Figure 3-9, it is once again important to remind readers that scores 
do reflect developmental progress; however, exiting third graders* grade equivalent scores are within third 
grade ranges. This implies that, as entering fourth graders, they may be considered "behind." 

Figure 3-9 



ARE K-3 STUDENTS "ON-TRACK?": THIRD GRADE STUDENTS' ITBS PERFORMANCE 




Raiding Lingiaf* Muh Compotttfl 

EE) FY 1989*90 □ FY 1990-91 



FY 1989-90 Third Grade Student* (Readinr n » 550; Language n = 542; Math n = 549; Composite n = 537) 
FY 1990-91 Third Grade Students (Reading 497; Language n = 498; Math n » 501; Composite n = 483) 



With respect to overall ITBS findings, it is encouraging to find increases in reading and language 
skills. Evidence discussed earlier in this chapter indicates that language proficiency is the key "predictor" 
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of academic success. It is also the stated emphasis of most programs. While many programs also 
addressed math skills, the emphasis was on using manipulates and developing mathematical concepts 
not easily captured by pencil-and-paper tests. Downward trends in math might be explained by the 
incompatibility of such curricula with standardized tests. Nevertheless, math programs should be further 
scrutinized, and educators should verify that math skills have been addressed adequately before children 
exit third grade. 

K-3 Student Promotion/Retention: Repeating a grade, in theory, is supposed to provide a child 
with a "second chance" to master skills. But recent research (Center for Policy Research, 1990; Shepard 
& Smith, 1989), discussed in the Arizona At-Risk Pilot Project FY 1989/90 Project Report (Bierlein et 
al., 1990), illustrates that retaining, or "flunking/ children is of questionable value: if any gains are 
made during the repeated year, they are short-lived. Further, it has been found that children who have 
been retained are more likely to drop out of school later in life than children who have not been retained. 
Considering both the costs of educating a child for one additional year (or more), and evidence suggesting 
that retention does not provide long-term benefits, alternatives to retention might be more cost effective. 

One hypothesis regarding retention is the following: the greater the use of educational alternatives 
to providing remediation, the less the need for retention as an educational solution. Knowing that many 
at-risk pilot sites consciously attempted to use research-based alternatives to retention, Morrison Institute 
evaluators investigated the incidence of retention at these sites. Findings through FY 1989-90 indeed 
showed a decline in retention rates during the period of at-risk pilot program implementation (Bierlein 
etal., 1990). 

During FY 1990-91, not only were retention data collected, but evaluators also explored whether 
or not at-risk programs directly contributed to lower retention rates. This was accomplished, in part, by 
the administration of a survey on retention (cf. Chapter 2). This survey examined the comparability of 
district-reported retention data by asking districts for definitions and examples of retention, as well as 
their criteria for making retention decisions. Responses to these questions were enlightening. For the most 
part, pilot districts did not haw spec[fic retention policies. Moreover, no standard criteria existed among 
districts (e.g. , a criterion aso. :as attendance was not consistent among schools or districts), and much 
variation existed in the grade levels offered. 

Several findings were of interest. First, almost all districts reported compliance with Arizona 
Revised Statute 15-701 (C) which states: " . . .a teacher shall determine whether to promote or retain a pupil 
in grade. . . . " Yet while state policy provides general guidelines, each district has been left to interpret the 
policy independently. As a result, most districts do not have specific policies, but, rather, general 
procedures (e.g., "When it becomes apparent that a student will have difficulty meeting the promotion 
standards of the grade level, both the student and the parent should be immediately advised..."). 

Second, although teacher recommendation was reported as the predominant criterion for retention, 
relatively few districts specified what criteria teachers used to justify their decisions (e.g., class 
performance, attendance, mastery of Essential Skills, ITBS scores, social-emotional maturity). For the 
few districts/schools that did offer specific criteria, the number of criteria, and their weight in the 
decision-making process, varied widely. For example, some districts reported that retention could be 
considered if a child's attendance was less than 90 percent of the state requirement 10 ; others specified 
75 percent as their cut-off point. 



Arizona Revised Statute 15-341(A) mandates that pupils shall attend school "for a period of not less than one-hundred seventy 
days of school or its equivalent.../ 
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Third, and more basic to the difficulties of the analysis, districts did not share a common 
definition of retention. For example, six districts offered transition or developmental grades during FY 
1990-91 as part of their at-risk program. Most districts (38 of the 41 districts responding) said they 
considered a child to be retained if that child repeated the same grade or was placed in a transition 
classroom. Two of the six districts offering transition or developmental grades, however, said they did 
not consider placements in these classes as retentions. These two districts did have unique programs 11 ; 
the point is, however, that children who would be considered retained in most other districts were not 
considered retained in these districts. Given the high mobility rates noted for the at-risk districts, inequity 
in retention practices could be cause for concern. i 

The retention survey also addressed the question of whether or not retention policies had changed 
since at-risk program implementation. Responses were analyzed with reference to existing district/school 
retention policies. In part, Morrison Institute evaluators wished to determine whether or not changes in 
retention rate were attributable to policy changes stemming from at-risk program implementation. 

Over half of the 42 pilot sites (23 districts) responded that there had been changes since the 
implementation of the K-3 at-risk program which had affected both policy and practice. Doc 'mented 
policy changes were submitted by two districts. In one, the principal "... mandated that no one be 
retained../ and in the other a "no retention policy for all kindergarten students..." was implemented. 
Seven additional districts indicated that, as a result of the implementation of the K-3 at-risk program, they 
were currently reviewing, evaluating, revising, and adopting retention/promotion guidelines and policies. 
Also, three schools attributed the decrease or elimination of their developmental and transition grades to 
the implementation of the at-risk program. Two of these added a full-day kindergarten as an alternative 
to the (eliminated) transition grade program. 

Respondents further noted that at-risk programs had encouraged their districts to explore 
educational alternatives (e.g., "Schools are now being encouraged to consider the research on 
retention..."), especially because teachers had become more aware of the research on retention. In order 
to prevent retention teachers had sought additional training, employed new teaching methods, expanded 
remedial programs, attempted to involve more parents, and paid more attention to individual learning and 
cultural differences. 

Have retention rates continued to decrease over time? Figure 3-10 illustrates the average retention 
rates for grades K-3 over the last four years (1987-88 to 1990-91). As shown, retention rates have 
continued to decline. While decreases may not be directly attributable to at-risk programs, evidence from 
the retention rate survey shows that, in some districts, program implementation was a contributing/actor. 



11 One district offered transition grades between kindergarten and first, first and second, and second and third grades. These 
were essentially "open-entry, open-exit" classrooms Uiat served children from both the grade before and after, depending on whether 
a child was advanced for his/her grade or experiencing difficulty. This district considered only the repetition of the same grade as 
a retention. The other district that did not consider placement in a transition grade as a retention offered a developmental first grade 
from which students could be promoted to either regular first or second grade. 
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Figure 3-10 



AVERAGE YEARLY RETENTION RATE (1987-91)* 
K-3 At-Risk Pilot Programs 



% retained 




Kind«rprttn 



* Ai reporter by district penonnel 



First S*cond 
I FY 1987-38 □ FY 1988-89 □ FY 1989-90 □ FY 1990-91 



Third 



Retention rate data were further examined by region and phase. Additional analyses were made 
to separate pilot schools from pilot districts. Regional analyses showed that rural districts retained the 
lowest percentage of students (1.3 percent), while urban districts retained the highest (3.5 percent). 
Reservation districts' rates were equivalent to the overall average (2.7 percent). Differences were also 
found between phases, particularly when pilot school data were examined separately from district totals 
(see Table 3-21). 



Table 3-21 



FY 1990-91 RETENTION RATES BY PHASE AND DISTRICT/PILOT SITES 


PHASE 


I 


II 


Population 


% Retained 


Population 


% Retained 


TOTAL DISTRICT 


25,614 


3.0 


21,683 


2.3 


AT-RISK PILOT SITES ONLY 


16,953 


2.2 


8,926 


3.7 



Table 3-21 shows that, when total district rates were compared, phase II districts had lower 
retention rates than did phase I districts. But when pilot school sites only were separated out and 
compared, he reverse was true: i.e., phase I school sites had lower retention rates than did phase II 
school sites. In fact, during FY 1990-91 phase II pilot sites served roughly half as many students as did 
phase I pilot sites, yet retained students at a rate more than one and one-half times greater. 
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This might be an indication that phase I at-risk programs made a positive impact, particularly 
considering that phase I at-risk program schools retained proportionally fewer students than did their 
overall districts (2.2 percent versus 3.0 percent). Certainly, formal changes in districts' retention policies 
were not an important factor since only two schools reported such changes. Nevertheless, uie limitations 
of the database prevent exact accounting, and this issue warrants further investigation (e.g., to determine 
if strategies used in the at-risk program schools might be successful alternatives to retention). 

District Self-Reported Outcomes: Morrison Institute encouraged all districts to submit self- 
evaluation data. The following discussion is not a comprehensive report on all district self-evaluation 
efforts. Rather, it illustrates some of the types of self-evaluation being done. 

Case Study #1: This rural district was one of the few to attempt a quasi-experimental study of 
at-risk program results. The district's "official" at-risk program was a K-l multi-year program which 
integrated at-risk and non-at-risk youngsters. In the official program, kindergarten (full-day) and first 
grade classrooms had the assistance of a part-time aide (to reduce the student-staff ratio). Apart from its 
official program, the district offered four other options for at-risk youngsters: 1) full-day kindergarten 
with aides, 2) full-day kindergarten without aides, 3) half-day kindergarten with part-time aides, and 4) 
half-day kindergarten with full-time aides. 

At-risk children in all five settings were pre- and post-tested using the Peabody Picture 
Vocabulary Test (PPVT: English version and Spanish version, when appropriate) as well as the 
Developmental Test of Visual-Motor Integration (VMI). At-risk students were compared with their non- 
at-risk peers in all five settings. Some results were inconclusive (i.e., the difference between student 
growth in classrooms with no aides, part-time aides, and full time aides). However, one finding bears 
note. When at-risk program kindergarten students (full day) were compared with non-program at-risk 
kindergarten students (half-day )-each with part-time aides-program students scored an average often 
months higher on the PPVT, and two months higher on the VMI than did non-program at-risk children. 
Also, at-risk program participants were only five months lower than their non-at-risk peers at the time 
of the post-test. In comparison, half-day kindergarten at-risk students were one year-one month lower than 
their non-at-risk peers. Additional studies are being conducted. 

Case Study #2: This rural district examined the effectiveness of an intensive reading tutorial 
program. Results were reported for seven at-risk first grade students targeted to receive this service. 
Based on the results from commercially prepared criterion-referenced mastery and exit level tests and 
conventional basal testing, five of the seven students were reported to be reading at or above grade level 
at the end of the year. In addition, one student dropped out of the program, but reportedly improved 
several levels prior to leaving. The district noted that significant teacher training on tutorial methods was 
provided in conjunction with the program and, in conclusion wrote: 

"[The tutorial] is a highly successful component of [the] K-3 At-Risk program. Documented 
success of students' reading accomplishments, teacher evaluations of students' progress, and 
parental enthusiasm make the continuance of the [tutorials] an essential attribute of [the district's] 
program. " 

Case Study #3: This urban district placed at-risk students in a full-day kindergarten. Pre- and 
post-tests were given in three areas: letter identification, writing vocabulary, and sentence dictation. Based 
on average stanine gains reported in all three areas, the district concluded that: 
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"The full-day kindergarten program is offering a wonderful opportunity for at-risk children to 
experience a successful start in school. ...The payoff fls] an academic readiness for grade one. " 

Case Study #4: This rural district focused specifically on ESL students. The objective: after two 
years, half of the students who had entered the program with no English proficiency would be operating 
in English at least 75 percent of the time. Of 16 students identified as ESL in 1989, 11 were assessed at 
the end of FY 1990-91 using a variety of standardized instruments as well as the judgements of the 
district's ESL aide and a classroom teacher. The district reported that 91 percent of the children -re 
operating in English at least 75 percent of the time. The district concluded: 

"All the students who entered this program are now speaking and communicating in English. 
These students have increased their self-esteem and their parents are proud of their achievements. 
The parents feel ihat [this] is an outstanding school. In fact, these students are now assisting their 
parents in translation of English into Spanish. One of the greatest strengths of this program has 
been the ability to pull together the parents and students, increase self-esteem and family pride. 
It has enabled our LEP and NEP students to keep up with their class/age peers and to be 
successful in their academic studies. " 



K-3 PROGRAM EVALUATION CONCLUSIONS 



PROGRAM IMPLEMENTATION CONCLUSIONS 

A comparison of survey ratings on implementation success/barriers, open-ended comments, and 
interview data yields several recurrent themes: 

• Poor/inadequate communication has adversely affected program implementation; many teachers 
want to have greater input. 

• Coordinating/integrating programs has been problematic; establishing school-community 
collaboration is an issue to be addressed, particularly in rural and reservation programs. 

• Program planning and "alignment" of philosophies appear to be on-going problems th?.t are 
intertwined with the issue of communication; in particular, many teachers want planning input. 

• Qualified and committed staff can either "make or break" a program; in order to produce 
qualified staff and update existing staff, adequate and appropriate staff trainin, ,4 required. 

• Strong program leadership and administrative support are essential for program success. 

While this section primarily has highlighted concerns regarding program implementation, it is 
important to recap the following positive findings: 

• Programs, in general, have been implemented as planned; they have served over 24,000 
students, established communication with over 6,000 parents while offering specific services to 
many others, and provided staff training for personnel at all 42 program sites. 
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• Of IS factors identified as affecting program implementation, all were rated more positive than 
negative by the more than 1,000 teachers/staff responding. 

• Open-ended question responses and interview data produced more positive than negative 
comments regarding program implementation. 

PROGRAM SERVICES CONCLUSIONS 

Student Services 

A comparison of survey ratings on student services, open-ended comments, and interview data 
yields the following themes: 

• Reducing student-staff ratios is considered an effective strategy for working with at-risk 
children; however, both more and more qualified staff are needed. 

• Full-day kindergarten holds promise for helping at-risk youth. 

• Tutorial programs can effectively offer individualized instruction; how or when tutorial 
programs should be implemented remains subject to debate. 

• Modifying classroom instruction-by using different curriculum materials and different 
instructional strategies-is well-received; modifications, however, may not be easily implemented 
without additional and appropriate materials. 

Parent Services 

A great deal of consensus exists between interview data and staff survey ratings of parent 
services. Open-ended question responses were not as clearly aligned with these findings, although they 
support the following conclusions: 

• Social events are effective in increasing parent involvement. 

• Efforts by staff to communicate in person with parents are most effective. That is oral 
communication (phone calls or home visits) is more effective than written communication (notes 
or materials mailed to the home). 

• Parent training opportunities-either through workshops or formal classes-are promising 
practices for parent involvement. 

• Although many perceive an overall increase in parental involvement, desired levels of 
involvement have not been achieved. 
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Staff Services 



Survey data, open-ended responses, and interview data on staff services provide a clear direction 
for staff development efforts to take. While staff generally agree that a variety of staff development 
strategies have worked well, they also identify on-going needs and offer specific ideas regarding what 
constitutes effective in-service. The following conclusions can be drawn from the data: 

• In-services should focus on classroom applications of specific teaching strategies that are 
relevant to at-risk populations; comprehensive follow-up should be included. 

• Teachers should be given time to process new teaching strategies and fully implement them in 
their classrooms. 

• Teachers benefit from communicating with their peers, and this can be achieved through 
common planning time, grade level meetings, and team meetings. 

• Staff in rural ;.nd reservation districts need better on-site staff development opportunities. 

• Classroom aides need more training on a variety of curriculum and instruction topics. 
PROGRAM OUTCOME CONCLUSIONS 

A comparison of teacher perceptions of program/student outcomes, student outcome data, and 
district-reported data, in conjunction with a study on student mobility, indicate the following: 

• Teachers perceive, and achievement data confirm, that students are making some developmental 
progress; however, the extent to which progress is directly attributable to at-risk programs cannot 
be objectively determined. 

• Although teachers believe, for the most part, that they are helping children stay "on track," 
many are also concerned that children advanced from grade-to-grade without appropriate skills. 

• Excessive student mobility is a substantial concern for some districts; even the best programs 
may not be able to make a difference if students do not remain in the program or school. 

• A trend toward decreased absenteeism has occurred since the inception of at-risk programs; this 
implies that children received "additional" instructional days. These additional dayr, combined 
with modifications in the types of services children receive, mean that at-risk pilot sites addressed 
both the quantity and quality of instructional time. 

• For the children represented by the K-3 cohort: 1) ITBS NCE results showed student progress 
in language and reading-the two areas emphasized within the at-risk programs; at the same time, 
ITBS NCE scores showed net declines in math; and, 2) ITBS GE scores showed that students 
were making steady developmental progress from year-to-year, but were still advancing "below 
grade level." 

• For consecutive groups of third grade children represented by the K-3 cohort, ITBS NCE 
results show progress in reading and language, and declines in math. 
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• Consistent results indicating net gains in reading and language are encouraging, particularly 
because these skills appeal' most predictive of academic achievement. 

• Consistent results indicating below-average developmental growth in math indicate a need to 
review math curricula to determine whether math skills are receiving adequate attention, or 
whether poor ITBS scores are a result of test-curriculum incompatibility. 

• Decreased retention rates have been observed in at-risk sites, and there is some evidence that 
these are associated with at-risk program implementation. 

• District-specific outcome data suggest positive impacts from programs and also illustrate the 
importance of locally-conducted evaluations to portray specific program effectiveness. 
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Chapter 4 

7-12 AT-RISK PROGRAMS; DESCRIPTIONS AND EVALUATION RESULTS 



This chapter includes a synthesis of information from each database compiled for the analysis of 
the 7-12 at-risk programs. Evaluation findings are presented in the first sections of the chapter; 
conclusions are discussed in the final section. 



DESCRIPTION OF THE 7-12 STUDENTS 

In order to better assess how well Arizona's at-risk pilot programs are serving '-risk students, 
Morrison Institute attempted to gain a clearer understanding of these students-their life circumstances and 
| the characteristics that make them at risk. This was accomplished through the administration of the 7-12 

Student Profit (cf. Chapter 2). 

I 7-12 AT-RISK STUDENT AND FAMILY CHARACTERISTICS 

■ A two-year comparison of profile data was conducted to assess annual t snds in student-reported 
I information. The comparison of both student and family data yielded a consistent portrait of at-risk youth 

being served by the 7-12 at-risk pilot programs. For student data, the only noteworthy difference between 
I FY 1989-90 and 1990-91 was in the category of work. Students surveyed in 1990-91 reported that they 

worked fewer hours than did students in 1989-90. 

| With the exception of a decrease in the use of the "no response" option, family informr^on was 

very similar for the two years of the study. A majority of students reported living with both natural 
parents. Most parents were said to have no more than a high school education. Students both years 

I reported that many mothers were unemployed outside the home and that a majority of fathers worked in 

"laborer/clerical" occupations. 

I FY 1990-91 demographic data were disaggregated by region (urban versus rural versus 

reservation) and by grade level (7-8 programs versus 9-12 programs). Tables 4-1 and 4-2 highlight the 

I regional analysis. Supporting documentation showing detailed breakdowns by grade level and by region 

' are available upon request from the evaluator. Regiona' findings show that, of the 1,627 students 

completing the profile, urban students comprised 19 percent, rural students 52 percent, and reservation 

I students 29 percent. Each region was treated independently under the assumption that respondents were 

representative of students served by pilot programs in each of the regions. Table 4-1 shows that both 
urban and reservation programs served approximately equal numbers of males as females, while rural 

| programs served twice as many males as females. The age distribution between regions shows that urban 

and reservation programs encompassed more students in the 11-14 year old age bracket than did rural 

- programs. Regarding "over-age" students (19-or-older bracket), reservation programs served the most, 

I followed by rural programs. 

I Urban and rural communities served approximately three minority students for every one white 

■ student, with Hispanic students the largest minority in both regions. Of the relatively few Black students 
served by the programs, most were in the urban areas. Data also suggest that minorities in rural areas 

I were more likely than their urban counterparts to maintain use of a native language other than English. 

As expected, reservation programs served almost exclusively Native American youth (95 percent). 
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Approximately one-third of reservation students lived in homes in which their native language was the 
primary language spoken. 

Data bear out what was already known about job opportunities for students: more urban students 
worked for pay than did either rural or reservation students. 



Table 4-1 



COMPARISON OF 1990-91 7-12 STUDENT CHARACTERISTICS BY REGION 




(Total N = 1627; Urban n = 313; Rural n = 841; Reservation n = 473) 




Gender 


Male 


Female 




No Response 




Urban 


55% 


45% 




<1% 




Rural 


66% 


33% 




<1% 




Reservation 


48% 


52% 




<1% 




Age uuiiriDUuon 


11 14 


H 1ft 


iy 


No Response 


uroan 








0% 


Rural 


26% 


68% 


6% 


<1% 


Reservation 


36% 


56% 


8% 


<1% 


Ethnicity 


White 


Hispanic 


Native 


Black 


Other/No 








American 




Response 


Urban 


24% 


62% 


3% 


8% 


2% 


Rural 


28% 


58% 


8% 


3% 


2% 


Reservation 


2% 


1% 


95% 


0% 


1% 


Primary Home Language 


English 


Spanish 


Native 


Other/No Response 








American 












Language 






Urban 


67% 


31% 


0% 


2% 


Rural 


54% 


42% 


2% 


2% 


Reservation 


57% 


1% 


36% 


5% 


Work for Pay 


Don't Work 


Work <20 


Work >20 


No Response 






hours/week 


hours/week 






Urban 


68% 


25% 


7% 


0% 


Rural 


72% 


16% 


12% 


<1% 


Reservation 


92% 


6% 


1% 


1% 



Table 4-2 shows that urban students' home environments were the least "nuclear family-oriented" 
of the three regions. Compared with students in other regions, fewer urban students lived with both 
natural parents, and more lived with stepparents, in single parent homes, or in "other" living 
arrangements (e.g., with relatives). In all regions, parents were most likely to be in "blue collar" 
occupations (i.e., laborer/clerical) and to have no more than a high school diploma. About one-third of 
mothers and one-quarter of fathers did not graduate from high school. 
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Table 4-2 



COMPARISON OF 1990-91 FAMILY PROFILE FOIi 7-12 AT-RISK STUDENTS BY REGION 


(Total N = 1627: Urban n = 313; Rural 


n - 841; Reservation n = 473) 




Family Structure 


Mother/Father 


Natural Parent/ 


Single Parent 


Uvea Alone 


Other/No 






Stepparent 






Response 


Urban 


43% 


17% 


25% 


3% 


12% 


Rural 


52% 


14% 


23% 


3% 


8% 


Reservation 


53% 


12% 


22% 


4% 


7% 


Parent Education 


< High School 


Hich School 


Some Pnlleae 


Pnllrtir 
vUUwKw 


Don't Know/ 










Graduate 


No Response 


• Mother 












urwuj 


7b 


£770 


mat 
16% 


9% 


14% 


Rural 


35% 


30% 


15% 


7% 


13% 


Reservation 


30% 


30% 


16% 


12% 


12% 


• Father 












Urban 


23% 


26% 


12% 


13% 


26% 


Rural 


29% 


27% 


14% 


9% 


22% 


Reservation 


27% 


24% 


18% 


11% 


20% 


Parent Occupation 


Not Employed 


Laborer/ 


Agricultural 


Professional 


LSKJH 1 rVIlUWf 






Clerical 








• Mother 












Urban 


31% 


32% 


2% 


21% 


14% 


Rural 


40% 


26% 


6% 


19% 


9% 


Reservation 


49% 


25% 


2% 


13% 


11% 


• Father 












Urban 


13% 


42% 


4% 


17% 


24% 


Rural 


11% 


36% 


13% 


21% 


19% 


Reservation 


19% 


49% 


3% 


10% 


19% 



Beyond the issue of regional differences, analyses focused on program differences related to grade 
level 12 . Specifically, analyses compared those programs being implemented in upper elementary or 
middle schools with those being conducted in high schools. For this analysis, 14 of 55 program 
components representing seven schools were classified as serving seventh and/or eighth grade students 
exclusively. Of the seven schools, two were classified as reservation, two as urban, and three as rural. 
Only one of the urban schools, however, was "inner city" urban--the other represents a suburban 
community that borders on and serves a rural area. Thus, certain demographic differences were found 
between 7-8 grade program students and 9-12 grade program students. 



I 12 This distinction was of particular interest given literature wliich points to the unique needs of (he upper elementaiy/middle 

school population and suggests that different t> pes of intervention programs may be of value for these children (e.g., Task Force 
on Education of Young Adolescents, 1989; Sherman, 1987). It should be noted that 7-8 programs may serve some older students: 

I conversely, 9-12 programs operating in unified districts may serve some younger students, including a small percentage of students 

ui grades 7-8. Despite some overlap in student age ranges, the primary distinction between component* offered at different grades 
is felt to be valid. Data are included in the separate data supplement document. 
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Students in the 7-8 programs represented 26 percent of the survey respondents. Compared to the 
9-12 programs, 7-8 programs served more females and almost exclusively minorities (93 percent), 
representative of Hispanic (42 percent) and Native American (45 percent) populations. This latter 
characteristic results from the locations of the studied 7-8 grade programs, and should not be interpreted 
as representative of 7-8 grade at-risk students in general. 

Many other characteristics of students in the 7-8 programs paralleled those of the 9-12 population. 
Higher percentages of 7-8 students, however, reported using a language other than English in their homes 
and, as expected, a lower percentage worked for pay. More 7-8 than f 12 grade program students also 
reported that they lived with both natural parents, and that their mothers did not work outside the home. 

PATTERNS OF AT-RISK INDICATORS AND ACHIEVEMENT 

An additional purpose of tiitf 7-12 Student Profile was to gather information regarding the at-risk 
status of students in the pilot programs. First, at-risk indicators were examined with respect to frequency 
of occurrence. Second, indicators were examined in relation to four academic achievement subgroups, 
and indicators were identified that distinguished high achievers from low achievers. All analyses were 
conducted by region, by grade level, and for the total population. 

What indicators best describe at-risk pilot project youth? 

Examining the data presented in the "TOTAL" column (Table 4-3), at-risk youth may best be 
described as teens who: 1) feel that work and/or other responsibilities interfere with school, 2) are 
involved in no extracurricular or community activities, 3) have previously been suspended or expelled 
from school, 4) have been held back at least once in elementary school, and 5) have parents who do not 
attend school functions. Table 4-3 also reveals regional variations in top-occurring indicators. 

Table 4-3 







% 7-12 POPULATION WITH INDICATOR BY REGION 






URBAN 




RURAL 




RESERVATION 




TOTAL 




(d - 313) 




(n - 841) 




(n - 473) 




(n - 1627) 




Interference w/tchool 


50 


Interference w/school 


51 


No telephone 


66 


Interference w/school 


53 


Suspended/expelled 


46 


No activities 


47 


Interference w/school 


59 


No activities 


44 


Dropout/"kick-ou' " 


42 


Held back >_ 1 grade 


42 


No activities 


41 


Suspended/expelled 


38 


Low par. participation 


40 


Suspended/expelled 


41 


Held back J> 1 grade 


34 


Held back >_ 1 grade 


37 


Sibling dropout(i) 


37 


Low par. participation 


39 


Low par. participation 


29 


Low par. participation 


36 


No activities 


37 


Dropout/*kick-out " 


37 


Suspended/expelled 


28 


Dropout/"kick-out" 


33 


Suicidal ideai/decdi 


35 


Sibling dropout(s) 


29 


Suicidal ideas/deeds 


27 


No telephone 


30 


Held back J> 1 grade 


30 


Suicidal ideas/deeds 


29 


Feels unsafe at home 


25 


Suicidal ideas/deeds 


30 


Convicted of a crime 


29 


Convicted of a crime 


23 


Sibling dropout(s) 


24 


Sibling dropout(s) 


29 


Feel* unsafe at home 


24 


Feels unsafe at home 


18 


Substandard home 


23 


Feels unsafe at home 


21 


Skipped school weekly 


21 


No telephone 


17 


Dropout/"kick-out" 


21 


Convicted of a crime 


21 


Drugs/alcohol weekly 


14 


Drugs/alcohol weekly 


16 


Convicted of a crime 


10 


Skipped school weekly 


14 


>. 3 schools in 2 years 


12 


Slapped school weekly 


15 


_> 3 schools in 2 years 


10 


Drugs/alcohol weekly 


13 


Has children 


12 


>_ 3 schools in 2 years 


13 


Has children 


9 


> 3 schools in 2 years 


12 


No telephone 


11 


Low parent support 


8 


Drugs/alcohol weekly 


8 


Substandard home 


11 


Low parent support 


8 


Has children 


7 


Skipped school weekly 


7 


Has children 


9 


Poor health 


7 


In U.S. < 3 years 


7 


Low parent support 


5 


Low parent support 


7 


Substandard home 


4 


Substandard home 


7 


In U.S. < 3 years 


3 


In U.S. < 3 yean 


5 


In U.S. < 3 years 


3 


Poor health 


6 


Poor health 


3 


Poor health 


5 
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Comparing regions, Table 4-3 shows that relatively greater numbers of urban youth reported the 
following indicators: have children, have a sibling who has dropped out of school, have dropped out 
themselves, indicate poor health, have seriously considered or attempted suicide, skip school, been 
suspended or expelled, been convicted of a crime, and have parents who are neither supportive of nor 
involved in their education. Of all three regions, urban youth were characterized by the most indicators 
that affected relatively more students. 

In comparison, relatively more rural youth reported that they were not involved in any 
school/community activities, attended school in the United States for three years or less, had enrolled in 
three or more schools within the last two years, had been retained at ieast once during their elementary 
education, and used drugs and/or alcohol on a weekly basis. Finally, relatively more reservation youth 
said they: feel responsibilities interfere with school work, live in homes that do not have year-round 
electricity and/or plumbing, do not have telephones, and do not feel safe and/or protected at home. 

From these data, a profile of a typical at-risk students in each region can be created. Urban at-risk 
youth appear to have considerably more behavioral indicators. Reservation students, in contrast, reflect 
"at-riskness" more on the basis of life circumstances over which they have little control. Rural at-risk 
students appear to have a combination of both behavioral factors and life circumstances. Such 
generalizations are presented not to create stereotypes, but to prompt discussion of the unique aspects of 
at-risk youth across Arizona. Clearly, different groups of students present different pictures of what it 
means to be " at-risk/ 

Grade level analyses suggest that 7-8 grade program students are not as seriously at-risk as their 
9-12 counterparts. As evidence, only three of 19 indicators on the survey affected higher percentages of 
7-8 students than 9-12 students. These dua imply that "at-riskness" is a developmental phenomenon to 
some degree, and that early intervention might forestall certain at-risk behaviors. 

How do indicators relate lo achievement? 

For each of the four academic achievement categories (mostly A's, mostly B's, mostly C's, less 
than C average), at-risk indicators were ranked in order of frequency (Table 4-4). Looking at the 
indicators that apply to 25 percent or more of students in each category, it is clear that as achievement 
levels drop, the number of indicators rises. For example, six indicators apply to 25 percent or more of 
the A students, while seven apply to B students, nine to C students, and 11 to the less than C-average 
students. In general, this pattern was consistent for regional and grade level analyses as well. 
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Table 4-4 



RANK ORDER OF 7-12 AT-RISK INDICATORS BY ACHIEVEMENT LEVEL (N « ld27)* 


Mostly A's 




Mostly B's 




Mostly C's 




Less than a C-average 


(n * 140) 




(n * 455) 




(n * 620) 




(n m 391) 




(% of 140) 


(% of 455) 


(% of 620) 


(% of 391) 


Interference w/school 


53 


Interference w/school 


51 


Interference w/school 


54 


Interference w/school 


55 














No activities 


54 














Suspended/Expelled 


52 


No telephone 


37 


No activities 


39 


No activities 


45 






Low par. participation 


32 


No telephone 


33 


Held back >. 1 grade 


41 






Suicidal ideas/deeds 


31 


Held back >. 1 grade 


31 


Suspended/Expelled 


41 


Low par. participation 


47 


Dropout/"kick-out" 


27 


Dropout/** lack-out" 


29 


Low par. participation 


34 


Held back _> 1 grade 


45 


No activitiei 


25 


Suspended/Expelled 


28 


Dropout/ "kick-out H 


31 


Dropout/ "kick-out" 


44 






Suicidal ideas/deeds 


27 


Sibling dropout(s) 


31 


Sibling dropouts) 


38 










Suicidal ideas/deeds 


30 


Suicidal ideas/deeds 


32 


Feeli unaafe at home 


23 






No telephone 


28 


Convicted of a crime 


28 


Suspended/Expelled 


23 


Sibling dropout(s) 


24 






Skipped school weekly 


28 


Held back >. 1 grade 


19 


Feels unaafe at home 


21 






No telephone 


27 


Sibling dropout(s) 


19 


Convicted of a crime 


17 


Feels unaafe at home 


22 






Drugs/alcohol weekly 


15 


Low par. participation 


15 


Convicted of a crime 


21 






Convicted of a crime 


15 


Substandard home 


12 


J> 3 schools in 2 yean 


15 


Feeli unsafe at home 


22 


Haa children 


14 


J>3 schools in 2 yean 


11 


Skipped school weekly 


13 


Dmgt/alcohol weekly 


19 


j> 3 school* in 2 yean 


11 


Drugs/alcohol weekly 


11 


Drugs/alcohol weekly 


13 


> 3 schools in 2 years 


11 


Substandard home 


10 


Haa children 


11 


Substandard home 


12 


Low parent support 


11 


Skipped school weekly 


10 


Skipped school weekly 


6 


Has children 


9 


Substandard home 


9 


Low parent support 


7 


Low parent support 


6 


Low parent support 


7 


Poor health 


9 


Poor health 


6 


In U.S. < 3 yean 


6 


Poor health 


6 


In U.S. < 3 years 


7 


In U.S. < 3 yean 


4 


Poor health 


4 


In U.S. < 3 yean 


5 


Haa children 


6 


Lines divide indicator* by quartile (e.g., 0-24%; 25-49%); 27 students are included in the total 


N who did not rate their 




achievement level 

















Indicators were next examined to see which, if any, distinguished between high and low achieving 
at-risk students. Inc tors were identified that increased in frequency across categories 13 . These 
indicators are hypothesized to be more associated with achievement than other indicators more equally 
distributed across performance categories. The indicators that were identified to be more pervasive among 
low achievers than high achievers in the total 7-12 population are as follows: 

• Student is not involved in any school/community activities 

• Student has been held back at least one elementary grade 

• Student has sibling(s) who has/have dropped out of school 

• Student has previously dropped out of school 

• Student has been suspended or expelled from school 

• Student has been convicted of a crime 
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11 Increases were examined regardless of the degree of increment. The use of this criterion limit* the analysis as there are certain 
indicators that appear to distinguish higher achieving at-risk youth (e.g., A and B students) from lower achieving students and other 
indicators that may distinguish C or above students from less than C students. 
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Indicators that distinguished among levels of achievement by region and grade were also 
identified. Regional results indicated that only one indicator showed a trend for urban students: 
responsibilities interfered with school. In contrast, four indicators appeared as trends in the rural areas: 
responsibilities interfered with school, involved in no activities, was suspended or expelled, and had been 
convicted of a crime. Finally, reservation low achievers appeared more at risk than high achievers with 
respect to six indicators: involved in no activities, was suspended or expelled, dropped out of school, was 
held back one grade or more, and had a brother or sister that dropped out of school. For the very few 
students to whom it applied, recent immigration to the United States was also more prevalent among 
reservation low achievers than high achievers. 

Grade level analyses show that most high and low achieving 9-12 students were equally at risk 
for a majority of indicators. Only two indicators show a trend across grade performance at the 9-12 level. 
Low achieving high school students tended to be less involved in activities and to have a 'bling who 
dropped out of school. In contrast, nine indicators emerge as descriptors of performance for 7-8 grade 
students. Low achieving junior high students were more likely to have been held back at least one grade, 
to have been involved in no activities, to have parents who participate in few (if any) activities, to reel 
unsafe at home, to have attempted or seriously considered suicide, to use drugs and/or alcohol on a 
weekly basis, to have been expelled or suspended, to have previously dropped out of school, and to have 
been convicted of a crime. 

In sum, this analysis reveals that at-risk indicators are distributed across achievement levels 
among certain pilot program students (e.g., urban students; 9-12 program participants). These students 
are more uniformly at risk, regardless of achievement. Other groups (e.g., 7-8 grade program 
participants) demonstrate that as at-risk indicators increase, academic success decreases. 

Vie significance of this analysis is embedded in an hypothesis that indicators with trends across 
achievement are potentially predictive of academic performance. Given the findings presented, intervention 
efforts might best be focused on reducing the influence of those at-risk indicators potentially predictive 
of failure. In other words: an ounce of prevention might be worth a pound of cure. 



DESCRIPTION OF THE 7-12 PROGRAMS 

Arizona's 7-12 at-risk pilot programs have provided both "preventive" interventions for those 
students for whom academic failure seemed imminent, and "curative" interventions for those who had 
already experienced failure. As of FY 1990-91, there were 13 at-risk pilot programs (11 "phase I" and 
two "phase II") funded at the 7-12 level. One of the programs, however, was a consortium comprised 
of eight school districts and a county alternative education center. Therefore, 21 individual sites were 
actually involved in the 7-12 Arizona At-Risk Pilot Project. These 21 programs are described in terms 
of student delivery systems and services provided to students, parents, and staff. 

STUDENT DELIVERY SYSTEMS AND SERVICES 

In an effort to capture the diversity of the 7-12 at-risk pilot programs, individual programs were 
classified on the basis of two critical attributes: 1) the degree to which at-risk students were served 
separately from their non at-risk peers (as a homogeneous group), and 2) the degree to which services 
supplanted versus supplemented "traditional" programs of study. Using these criteria, ?, continuum of 
programs/activities emerges. At one end are totally independent and relatively self-contained programs; 
at the other end are programs completely integrated within the regular school into which at-risk students 
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are "mainstreamed." Four types of delivery systems along this continuum will now be described 
(Appendix B; Table B-l depicts the classification of districts' delivery systems.) 

Types of Delivery Systems 

Alternative Schools/Programs: Alternative schools/programs generally offer at-risk students an 
independent program of study in which to earn credits and complete school. Programs have designated 
staff members, offer a range of academic and vocational course work commensurate with "traditional" 
school offerings, intensify efforts to provide social services, and generally employ a variety of methods 
to capitalize upon teaching/learning styles. Programs may have flexible hours, from early morning to late 
evening, to accommodate students with various scheduling difficulties. For the most part, these programs 
are located on their own campuses and supplant participation in other school programs. 

Ten programs axe best described as alternative schools/programs: Apache Junction's Alternative 
School, the Central Arizona Alternative School, Mammoth/San Maimers El Camino program, Marana's 
Alternative School, Nogales' STRIVE program, the Pima County Detention Center School, Sanders 
Alternative School, Somerton's Alternative School, Superior's Alternative School, and Tucson's PASS 
program. 

Project CAPE, located in the Pima County Juvenile Court Center, represented a unique alternative 
school model in that detained youth received academic screening and educational services while they were 
in the detention facility. A transition coordinator served as a liaison with local school districts, juvenile 
probation, and the Department of Corrections, ensuring that a student's academic status and progress 
were reported to the appropriate organization when the student left the CAPE program. Dropout retrieval 
was central to the program-detained youth who dropped out of school were reenrolled through CAPE. 

Schools-within-schools (SWS) M : The primary characteristic of these programs is that 
homogeneously grouped at-risk students are served together for a block of time on a regular basis. Like 
alternative programs, these programs usually employ a greater variety of instructional methods for the 
purpose of individualizing instruction. These programs may incorporate academic, vocational, and support 
services, but do so in a more structured manner than do alternative schools. Unlike alternative schools, 
these programs generally do not offer flexible scheduling and students typically participate in the 
"regular" school program for at least part of the day. 

Nine programs in nine districts are best described as schools-within-schools: Apache Junction's 
Intervention School, Casa Grande Elementary's alternative program, Coolidge's BEARS program, 
Creighton's CARE class, Ganado's Applied Technology class, Maricopa's alternative program, Nogales' 
Middle School alternative program, and Somerton's "Itinerant Program" (two classes). 

Classes/Labs/ Activities: The primary characteristic of these "programs" (most often viewed as 
program components) is that they are supplemental to the regular school program. At-risk students 
participate in the regular curriculum for the greater part of the day but receive "extra" or "special" 
services in a class, on a pull-out basis, after school, or during the summer. Of 35 components in this 
category, 17 provided additional academic assistance (e.g., tutoring, computer labs, "special" classes), 
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14 Thin category encompasses programs that were classified as both "on-site alternatives" and "special classes" during FY 1989- 
90. Throughout 1990-91, it has become increasingly apparent that the distinction made last year between on and off-site alternative 
programs and "special classes" is not * "critical attribute" in distinguishing effectively between delivery systems. 

68 

•- 7ti 



seven are in the area of vocational services (e.g., work experience programs, high tech lab), and 1 1 were 
in the area of social/support services (e.g., support groups, clubs, counseling). 

School Reform Efforts: Two at-risk sites implemented total school reform efforts. In this case, 
there were not "programs' 1 designed to serve at-risk students, but rather school improvement efforts 
targeting all students equally. Kayenta implemented a four-period day; Casa Grande Union High School 
initiated major staff development efforts. Both of these districts, however, also maintained certain 
classes/labs/activities more suited to the needs of at-risk students. 



* * * * * 



The distribution of these delivery systems across regions is depicted in Table 4-5. This table 
shows that urban districts, for the most part, implemented discrete program components under the 
auspices of their at-risk grants. Notably, urban districts offered the most social/support components of 
all regions. Rural districts, in contrast, offered more "holistic" interventions (i.e., alternative schools and 
schools-within-schools) which integrated academic, vocational, and support services. Reservation districts 
tried a variety of interventions, but primarily offered discrete program components. Notably, they are 
offering the most components focused on vocational activities. 



Table 4-5 



TYPES OF 7-12 DELIVERY SYSTEMS BY REGION 



REGION 



Alternative 
School 
Program 



School 
Within 
School 



Academic 
Class/Lab/ 
Activity 



Vocational 
Class/Lab/ 
Activity 



Support 
Class/Lab/ 
Activity 



School- 
wide 
Reform* 



Total 0 of 
Programs/ 
Components 



Urban/ 
Suburban 

Rural 

Reservation 



2** 

7 
1 



18 

23 
14 



Total # Delivery 
Svstems 



10 



17 



11 



55 



* Only one of two school-wide reform effort* is presented on this table (Kaycnta's "Four-Period Day") because only 
these students were specifically surveyed regarding this effort. 

** One of these programs is the Pima County Detention Center, which has been excluded from some analyses presented 
in subsequent sections of this chapter. 



Types of Student Services 

Each of the programs summarized above was developed in response to the requirement set forth 
by H.B. 2217 (1988) that required districts to: 

"... establish a demonstration education and training program that specifically addresses the needs 
of the secondary at-risk pupil through alternative programs and activities that provide academic 
and vocational training as well as support services for dropouts and potential dropouts." 
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As should be clear, some districts developed alternative programs that integrated and directly 
delivered academic, vocational, and support services; other districts built a "program" around one 
activity- either academic or vocational or support. Thus, in keeping with the specifications of H.B. 2217 
(1988), a variety of alternative programs and activities were, indeed, implemented under the auspices of 
the at-risk demonstration projects. 

A question remains. What types of academic, vocational, and support services have been 
provided? As described in detail in the Arizona At-Risk Project FY 1989-90 Project Report (Bierlein et 
aL, 1990, pp. 60-61) and outlined above, each program attempted to integrate academic, vocational, and 
support services in unique combinations. For purposes of program evaluation, however, frequently 
occurring types of services in each area v/ere identified as follows: 

Academic/ 

Instructional Services: 9 Small group instruction 

• One-on-one tutoring 

• Computer-assisted instruction (CAI) 

• Self-paced curriculum 

Vocational Services: • Applied academics 

• Employability skills 

9 Career awareness activities 

• Vocational/occupational training 

• Work experience/Entrepreneurial activities 

• Vocational assessments/Individual Vocational Education Plans 

• High Tech labs 

Social/Support Services: • Life/study skills classes 

• Personal counseling (individual, group, family) 

• Formal student monitoring 

• Formal student mentoring 

• Social service linkages/referrals 

Essentially, these services can be viewed as a menu from which districts selected services to 
incorporate within their programs 15 . 

PARENT SERVICES 

Concurrently with H.B. 2217 (1988), State Board criteria for 7-12 at-risk pilot programs required 
"parental communication H in conjunction with direct student services. During FY 1989-90, four major 
strategies were identified as used by districts to involve parents. These same strategies were employed 
during FY 1990-91 and included the following: 1) increasing home/community outreach efforts through 
oral and/or written communication and formal home visits; 2) increasing opportunities for school-based 
'nvolvement focusing on school events to which parents are invited; 3) upgrading parent skills through 
workshops and/or classes; and 4) providing counseling/social services through which parents are served 
and/or referred to social service agencies for help with family problems. 



15 Note, however, that thin list did not exist as such when programs began. 
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STAFF SERVICES 



Although staff development was not an area of emphasis for 7-12 at-risk programs, many districts 
provided some staff development activities in support of their programs. Additional training was provided 
through: 1) workshops/in-service, 2) conferences and academies, 3) formal classes, 4) formal program 
meetings, and/or 5) school visits and observations. 

INTEGRATION OF SCHOOL AND COMMUNITY SERVICES 

To the degree possible, 7-12 at-risk pilot programs were encouraged to coordinate services with 
other programs and local agencies. State Board criteria included written agreements between schools and 
cooperating agencies. Since FY 1989-90, inter-agency collaboration has been reported and verified 
between programs and the following: chambers of commerce, Indian tribal agencies and organizations 
(e.g., chapter houses), civic organizations, local businesses, local and county government agencies, 
private and public health care agencies, juvenile justice systems, community and family resource centers, 
vocational programs, community colleges and universities, day care centers, mental health organizations 
and providers, law enforcement agencies, and private counselors. 



PROGRAM EVALUATION ACTIVITIES AND RESULTS 

It should be clear that there are several frameworks within which to describe 7-12 programs. As 
of FY 1990-91 , 13 pilot programs encompassed 21 sites and at least 55 program "components," of which 
14 specifically targeted 7-8 grade students. Programs have been classified by region-urban, rural, and 
reservation-and by type of service delivery. Three kinds of services-academic, vocational, and support- 
were offered in unique combinations. While some programs directly incorporated all three service areas 
in one or more components, others sought to meet student needs through referrals and linkages with other 
school, district, and/or community-based programs. In sum, 7-12 programs have been diverse and multi- 
faceted. 

Within this context, program evaluation efforts need to be clearly delineated. First, Morrison 
Institute has focused on examining what constitutes "effective" program implementation. Second, both 
student delivery systems and discrete services (i.e., academic, vocational, and support) have been 
scrutinized, as have parent and staff services. Third, program outcomes have been analyzed. All three 
contribute to an understanding of "what works" for at-risk 7-12 students. 

PROGRAM IMPLEMENTATION 

Evaluation efforts were aimed at determining not only the extent to which programs wero 
implemented as planned, but how and how well they were implemented. To this end, comprehensive 
descriptions were compiled documenting program aspects such as planning efforts, student identification 
and placement criteria, staffing patterns (including turnover), and communication efforts among and 
between program staff and others, including school and district personnel, parents, and community 
members. 

Program Participation 

Each program was asked to submit participation data for students, parents, and staff. 
Furthermore, districts were asked to report formal at-risk program linkages by agency and type of service 
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(e.g., academic, vocational or social/support), as well as the numbers of students served through these 
linkages. All participation data were cross-checked with program descriptions as part of a process to 
verify that program services had indeed been implemented as planned. Participation data indicate that all 
21 sites offered student services. All sites also attempted to improve parent communication and 
involvement, and provide opportunities for staff training and professional development. Participation data, 
aggregated for all programs and averaged for the year are presented in Table 4-6 (see also Appendix B, 
Tables B-2 through B-7). 



Table 4-6 



7-12 AT-RISK PROGRAM PARTICIPATION DATA FOR FY 1990-91* 


Student Services 




• StudenU served by at-risk programs (unduplicated estimate) 


9,385 


• Students served through formal program linkages 




(may include duplicated counts) 




— Through 22 academic/instructional linkages (e.g., community colleges) 


815 


Through 43 vocational linkages (e.g., JTPA) 


1,116 


~ Through 79 social service linkages (e.g., counseling, DES) 


3,055 


Parent Services (unduplicated estimates) 




• Increased home/community outreach efforts 




- written communication 


1,532 


~ verbal communication 


731 


- formal home visits 


371 


♦ School-based involvement (i.e., events) 


1,183 


• Upgrading parent skills (i.e., workshops/classes) 


288 


• Counseling/social services 


182 


Staff Services (may include duplicated counts) 




• Workshops/in-service 


1,068 


• Conferences/ Academies 


166 


* Formal classes 


166 


• Formal at-risk program meetings 


392 


• Schools visits/observations 


23 


'Duplicated counts arc a result of the fact that the same student or staff member may have participated 


in two or more 


significantly different kinds of services or activities 16 . 





Whenever possible, numbers reported arc estimated unduplicated counts. The student number requires some explanation. 
An a\er,igj of 7,792 students were served each semester by one or more of the 55 program components encompassed by the 7-12 
At-Risk Project. Some students served in spring semester, however, were also fall semester participants. Accounting for these 
"cany*over H students, the unduplicated estimate of 9,385 is calculated as follows based on the data reported in Table B-2: 

a. Fall semester H served - It leaving = students remaining in the program (7722 - 1524 = 6198) 

b. Spring semester # served - fall students remaining = "new" spring students (7861 - 6198 = 1663) 

c. Fall semester # served + H new H spring students = total students served (unduplicated estimate: 7722 + 1663 = 9385) 
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Quality of Implementation 



While data indicate that services were implemented, they do not reveal how well services were 
implemented. This latter issue, "quality of implementation/ was addressed in several ways. At the onset 
of the 1990-91 school year, program directors were asked to list the aspects of their programs that they 
felt either contributed or constituted a barrier to the successful implementation of their programs. This 
list of successful and unsuccessful practices was included in the Arizona At-Risk Pilot Project Report for 
FY 1989-90 (Bierlein et. al., 1990) and will not be repeated here. The list, however, was used to generate 
items for inclusion on the FY 1990-91 7-72 Teacher Survey. 

Fifteen salient items were identified by program directors as elements that could either "make or 
break" a program. An assumption was made that an end-of-year analysis of ratings would be useful in 
determining the extent to which potential barriers to success had been overcome during FY 1990-9 i. 
During spring 1991, survey respondents were asked to rate each of the 15 items on a 4-point Likert scale. 
The ratings of these elements are presented in rank order in Table 4-7. 

Table 4-7 indicates six key program implementation issues that continue to be perceived as 
barriers; school-district philosophies toward at-risk students, funding, school and community 
collaboration, pre-service and in-service staff training, and communication. 

Table 4-7 



FACTORS AFFECTING 7-12 PROGRAM IMPLEMENTATION (N= 982)* 

Very much a contributor to success: 4.0 



Staff commitment to working with at-risk student* 3.08 

Administrative support for new programs/change 2.93 

Evaluation assistance provided by Morrison Inatitutc/aite cvaluatora 2.77 

Strong program leadersliip 2.73 

Program assistance provided by the Arizona Department of Education 2.70 

Staff commitment to working with parents of at-risk students 2.68 

Teacher "buy-in" for new programs/change 2.55 

On-going dialogue/collegiality among teachers on how to assist at-riak students 2.55 

Integrated school-district jjlarj for meeting needs of at-risk students 2.54 

"Alignment" of school-district philosophies toward at-risk students (e.g., testing, curriculum) 2.50 

Availability of funds/resources earmarked for at-risk 2.48 

School and community collaboration in meeting student/parent needs 2.30 

Quality of in-service w/respect to at-risk issues 2.29 

Quality of pre-service w/respect to at-risk issues 2.28 

Clear communication to all staff re: program objectives, implementation, and refinements 2.27 



Very much a barrier to success: 1.0 

Scale: 1.0 = Very much a barrier; 2.0 = somewhat a barrier, 3.0 = somewhat a contributor; 4.0 = very much a 
contributor. [NOTE: Teachers were directed to consider characteristics a barrier if they were NOT ir. place to the extent 
that they "should" be.] 



These results were further analyzed using chi-square tests of significance to compare respondents 
by four variables; region, role, grade level affiliation, and level of program awareness. All chi-square 
tests were significant. Regional results show that rural teachers were more positive than expected (given 
expected frequencies defined by the chi-square tests). In comparison, urban and reservation teacher 
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responses were more negative than expected. This suggests that urban and reservation respondents were 
less satisfied with program implementation than rural teachers. 



Results analysed according to the respondent's role in the district/school show that administrators 
were more positive tfian their teaching/specialist staff. In the latter category, vocational educators were 
the only group to be more positive than expected. In terms of grade level affiliation (grades 7-8 only, 9- 
12 only, or both 7-12), the relatively few 7-12 respondents-and these might include more administrators- 
were consistently more positive than their 7-8 colleagues who, in turn, were more positive than their 9-12 
colleagues. It might be inferred, therefore, that academic instructors and 9-12 grade respondents were 
less satisfied with program implementation than their colleagues. Finally, results by level of program 
awareness (i.e., program staff versus "aware" and "unaware" non-program staff) indicate a continuum 
of responses that ranges from program staff (most positive) to unaware non-program staff (least positive). 

Survey respondents were also given the opportunity to make open-ended comments on program 
implementation. Of 980 respondents, 45 percent remarked on one or more program strengths in the areas 
of academic/instructional services, vocational services, social/support services, and the program in general 
(the latter encompassing parent and staff services and program implementation issues). An additional 43 
percent recommended changes in one o; more areas. Figure 4-1 indicates that of the 568 comments made 
regarding program strengths, most pertained to academic/instructional services, whereas most 
changes/recommendations were made for the "program in general." 

Figure 4-1 

7*12 PROGRAM STRENGTHS AND RECOMMENDED CHANGES 



7-12 Programs: 
Strengths 

(568 comments) 




Academic/ Vocational 
Instructional Services 
Services 



Social/ 
Support 
Services 



Program 
In General 



7-12 Programs: 
Recommended Changes 
(473 comments) 




Academic/ 
Instructional 
Services 



Vocational 
Services 



Social/ 
Support 
Services 



Program 
in General 



Taking a closer look at the "program in general/ Figure 4-2 illustrates both the strengths and 
recommended changes made in this area. Strengths most often were reported in relation to staff (42.2 
percent of all comments made). Comments primarily focused on the qualifications and/or commitment 
of staff. Specific implementation policies and/or procedures were also felt to be strengths (29.8 percent 
of all comments). Such things as good planning and coordination of services were noted, particularly in 
relation to producing student outcomes (e.g., "helps kids feel better about themselves "). 
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Of the comments made under "recommended changes/ 17.4 percent actually reflected positive 
attitudes toward programs, expressing needs to expand them and/or hire additional staff. Otherwise, 82.6 
percent of the comments focused on concerns. Major areas of concern regarded specific procedures and 
communication. Procedures needing improvement included student identification and placement 
techniques, funding mechanisms, and the coordination and integration of services within districts and the 
community. Concerns regarding communication were diverse but included: poor communication due to 
misaligned philosophies; poor "PR" by program staff on available services; inadequate sharing of 
information between program and non-program staff, including district office personnel and school boards 
who-according to several respondents-did not aiways have access to "appropriate" information. 

Figure 4-2 

7-12 PROGRAM IMPLEMENTATION STRENGTHS AND RECOMMENDED CHANGES 



Program in General: 


Program in General: 


Strengths 


Recommended Changes 


(568 comments) 


(224 comments) 


Parent Programs Communication 
so * 10.0% 


Suff Parent Programs 
12.5% 


Suff / 
42.2X 1 


IpH^, Leadership/ 

p Prg Mgt 
fjlilll 13.0% 


Programs ^^^^^^^^^^ Communication 








Planning/Procedures 
29.8% 


Planning/Procedures Leadership/ 
2U% Prg Mgt 



In addition to survey data, interviews with 326 persons (see Appendix B, Table B-9) also touched 
on program implementation. Parallel to the diversity observed in the open-ended comments, responses 
were mixed regarding the effectiveness of overall program implementation. This finding reflects the 
diversity of the school districts involved. 

Most respondents presented an overall sense that thftir programs had succeeded, and felt good 
about their impacts on students while they were in the program. 

• Said some urban students: "Without (the program) we would just h?iig out and make 
trouble." An urban parent said, "If (my child) had come here and betn thrown into the 
regular system, I don't know what would have happened to her... Son&h?«g happened 
to gei (my child) on track. I don't know exactly what, but it wzs this program." An 
urban community member echoed these sentiments, saying, "Of all the entities I've been 
in contact with, I've been most impresseu with what (this program has) done. Whatever 
they're doing, they're certainly doing it right." A reservation parent added, "It has really 
touched my kid." 
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Nevertheless, several issues were mentioned consistently throughout the interviews. Lack of 
communication among staff was a repeated concern. While staff at some schools felt that teachers and 
administrators were well informed about their at-risk program components, many more respondents felt 
the opposite was true. They recommended better formal and informal contacts between program staff, 
non-program teachers, and administrators. 

Interviewees also called for better linkages between discrete program components, between 
programs and schools/districts, and between programs and the community. Several respondents noted 
a lack of follow-up on students making the transition from junior high to high school, and from 
alternative programs to mainstream programs. Students, they felt, could be "lost" once they left the at* 
risk programs. With respect to program-community linkages, staff from some urban districts reported 
improved community links as a result of at-risk programs. Examples of good linkages included job 
placement programs and family support services linked to social service agencies. In contrast, establishing 
good community linkages proved problematic for rural and reservation districts. 

• Said one reservation staff member: "Kids fall through the cracks without contacts with 
outside agencies." 

Staff from some smaller districts said that certain elements of their programs had been delayed, 
disrupted, or not implemented as a result of late funding from the Arizona Department of Education. 
Despite letters of intent to fund from ADE, actual funds didn't reach some districts until mid-year. 

• As one urban staff member said: "You cannot hire people in the middle of the third 
nine weeks of school. Other districts might be able to operate with a piece of paper, but 
our district will not allow us to operate without an approved budget because we cannot 
afford the liability." 

Staff at some schools felt that program management had been good, and that systemic change 
was definitely occurring within their districts. Staff from several other schools, however, said their 
programs lacked focus and direction, and that planning was slipshod. Several staff called for more 
planning meetings to discuss progr.^m issues, and more support from administrators and school boards. 

• Said one urban staff member: "... the program has had no direction. As far as I know, 
the program... is in limbo. There is no director." Regarding district management, another 
teacher said, "They're reinventing the wheel without consulting us first." 



As with K-3 programs, leadership, p: jgram management, and communication issues ?t the 7-12 
level appeared inextricably intertwined. Concerns voiced during the past two years indicate that a lack 
of stable : .adership creates discontinuity in terms of program and school philosophies, staff and school 
organization, and program implementation. Given these concerns, Morrison Institute investigated the 
extent of administrative staff turnover since the inception of each program (i.e., 1988-89 for phase I 
programs; 1990-91 for phase II programs). Table 4-8 depicts the results of this investigation and reveals 
that high rates of key personnel turnover are cause for concern-particularly in the more isolated and rural 
programs. 
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Table 4-8 





7-12 PROGRAMS: ADMINISTRATIVE TURNOVER* 


Programs (by Region) 


At-Risk Project Director 


DLsty School Administration 


Urban (S programs) 
Rural (4 programs) 
Reservation (4 programs) 


2 out of S 40% 

3 out of 4 75% 

4 out of 4 100% 


2 out of 5 40% 

3 out of 4 75% 

4 out of 4 100% 


TOTAL (13 programs) 


9 out of 13 69% 


9 out of 13 69% 


* Turnover data were extracted from Morrison Institute formative and summative evaluation reports: September 1990; 
January 1991; and June 1991. 



PROGRAM SERVICES 

This section examines the evaluation results pertaining to student, parent, and staff services. 
Student services were evaluated in terms of delivery systems and discrete academic, vocational, and 
social/support services. FY 1990-91 evaluation efforts focused on determining staff and client perceptions 
of services through surveys and interviews, as supplemented by external evaluations over time. 

Student Delivery Systems and Services 

Delivery Systems: Staff were surveyed regarding whether or not each of five types of delivery 
systems "worked": on and off-site alternative programs, special classes and/or course work, tutorial 
programs, and summer school 17 . Table 4-9 indicates that 7-12 staff perceived supplemental services 
(i.e., summer school, tutorial programs, and classes) as "working better" than on and off-site alternatives. 
Rating the effectiveness of delivery systems higher than their colleagues we*e: rural staff, grades 7-12 
staff (grades 7-8 staff are least positive), and vocational instructors and administrators. Finally, program 
staff rated delivery systems higher than did aware and unaware staff, respectively. 



Table 4-9 



DELIVERY SYSTEM H EFFECTIVENESS M : 
7-12 STAFF PERCEPTIONS (N * 980) 




Has worked very well (needs to be maintained as is): 4.0 


Summer school 


3.04 


Tutorial programs 


2.94 


Special classes/course work 


2.91 


Off-site alternative programs 


2.88 


On-site alternative programs 


2.81 




lias not worked at all (needs total revision): 1.0 


Scale: 1.0 = Has not worked at all; 2.0 = Has not worked well; 3.0 = Has worked well; 4.0 = Has worked very well 



Relating these to the FY 1990-91 revised classification scheme, alternative schools are parallel to "off-site alternative 
schools," schools-within-schools encompass "on-site alternative schools," and classes/labs/activities encompass "classes/course 
work," "tutorial programs, " and "summer schools" (as ail of the latter are supplemental in nature to the traditional course of study). 
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In contrast to teacher perceptions, Table 4-10 illustrates how at-risk students participating in 
various delivery systems rated their overall "effectiveness/ Table 4-10 indicates that alternative schools- 
perceived as least effective by teachers-were most positively perceived by student participants. Discrete 
academic and vocational "components," generally perceived in a positive light, were nevertheless least 
"effective" from the students' points-of-view. As a whole, at-risk students and 7-12 staff surveyed did not 
see delivery systems eye-to-eye. 

Table 4-10 



DELIVERY SYSTEM "EFFECTIVENESS": 

7-12 STUDENT PERCEPTIONS (N ■ 1,627) 

Has worked very well*: 4.0 



• Alternative schools (n=344) 2.90 

• Social/support clawes/labs/activitics (n=309) 2.68 

• SchooU-within-ichoola (n=253) 2.67 

• School-wide reform (n=100) 2.64 

• Academic classes/labs/activities (n=380) 2.60 

• Vocational classes/labs/activities (n=241) 2..'>6 



Has not worked at all*: 1.0 

* "Working well" in this case is operationally defined as having produced changes in student behavior and attitudes. 
Means represent the average student ratings of 14 individual program outcomes (i.e., means of means). Program outcomes 
will be further defined and elaborated upon in the Program Outcomes section of this report. 



Types of Services: An additional question in the evaluation of student services was: Do specific 
types of services appear to "work" better than others? In response to this question, 7-12 students and 
teachers were asked to rate specific strategies on a 4-point Likert scale. Teachers were asked whether or 
not each strategy "worked" in meeting the academic and/or social/emotional needs of at-risk students 
(i.e., should be maintained or discarded). Students were asked to rate how much specific strategies had 
"helped" them either academically ("to do better in school"), vocationally ("to prepare for a job or 
career") or in terms of social support ("to feel better about yourself). 

Recognizing that not all 7-12 programs addressed all three service areas in their at-risk programs, 
services were evaluated in the total school context. The rationale was that perceptions of "working" or 
"helping" should be reflected, regardless of where offered, i/the program offered the service directly or 
provided it through an appropriate linkage. 

Teacher and student ratings of services are presented in Table 4-11. This table indicates that both 
teachers and students rated academic/instructional services highest, social/support services second highest, 
and vocational services third. The rank order of specific services, however, varied between teachers and 
students. For example, teachers rated small group instruction as "working" best of the instructional 
strategies, while students viewed working at their own pace (self-paced instruction) as helping them the 
most. 
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Table 4-11 



7-12 TEACHER/STUDENT RATINGS OF STUDENT SERVICES* 


Teacher Mean Rating 
(N = 982) 


Student Mean Rating 
(N= 1627) 


Academic/Instructional Services 

• Small group instruction 3.12 

• One-on-one tutoring 3.09 

• Computer-assisted instruction 3.02 

• Self-paced curriculum 2.83 

Academic/Instructional Average Mean 3.04 


Academic/Ins tructioual Services 

• Self-paced curriculum 3.04 

• Small group instruction 2.75 

• Computer-assisted instruction 2.65 

• One-on-one tutoring 2.62 

Academic/Instructional Average Mean 2.77 


Vocational Services 

• Career awareness 2.95 

• Applied academics 2.90 

• Vocational/occupational training 2.80 

• Employability skills 2.79 

• Vocational assessment (IVEPs) 2.79 

• Work experience 2.75 

• High tech labs 2.57 

Vocational Services Average Mean 2.86 


Vocational Services 

• Applied academics 2.79 

• Career awareness 2.66 

• Employability skills 2.62 

• Vocational assessment (IVEPs) 2.33 

• Vocational/occupational training 2.20 

• Work experience 1.91 

• High tech labs 1.77 

Vocational Services Average Mean 2.33 


Social/Support Services 

• Counseling 2.96 

• Linkages/referrals 2.88 

• Life skills/study skills 2.87 

• Mentoring 2.76 

Social/Support Services Average Mean 2.88 


Social/Support Services 

• Study skills class(es) 2.62 

• Life skills class(es) 2.56 

• Counseling 2.40 

• Mentoring 2.36 

• Linkages/referrals 2.23 

Social/Support Services Average Mean 2.43 


* Ratings are derived from parallel, though not identical, 4-point Likert scales. Scores between 1.0 and 2.5 are generally 
negative (doesn't work/didn't help); scores between 2.5 and 4.0 are generally positive (do work/help). Items have been 
paraphrased for inclusion in this table. 



Analyzing staff survey results by region, rural staff rated all services-academic, vocational, and 
social/support-more positively than their colleagues. Reservation instructors tended to rate academic/ 
instructional services higher than did urban staff; urban staff rated vocational and social/support services 
higher than did reservation staff. Grade level analyses show that 7-8 staff and 9-12 staff were fairly 
similar in their responses to the effectiveness of academic/instructional and social/support services. Grades 
7-8 staff were not as positive regarding vocational services as their 9-12 colleagues. The relatively few 
7-12 staff members who responded consistently rated all services higher than their counterparts 18 . 

Analyzing student survey results by region, reservation students tended to be more strongly 
positive (i.e., rating services as "helped a lot H ) about all services than did their rural or urban 



Teacher survey results were also analyzed by the each respondent's primary role and level of program awareness. For all 
services, administrators and vocational instructors rate services as "working welT more frequently than academic instructors and 
specialists. Non-program, but awr<rc, staff were most positive regarding program services; unaware staff were least positive. 
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counterparts. Urban students rated vocational and social/support services higher than did rural students, 
Student responses analyzed by grade reveal higher student satisfaction among 9-12 students with 
academic/instructional services, and among 7-8 grade students with social/support services. Grade 7-8 
students indicated they received fewer vocational services than did their 9-12 peers; however, those 
students who did receive vocational services (7-8 and 9-12) rated their effectiveness similarly. 

One final analysis of student ratings examined what services they received in relation to how they 
received them. It was hypothesized that students in alternative schools, schools-within-schools, and 
school-wide reform efforts would reflect roughly equal satisfaction with all three types of services, since 
all were integrated/coordinated within these types of delivery systems. In contrast, students in 
academically-focused classes/labs/activities were not expected to highly rate vocational or support 
services; students in vocationally-focused components were not expected to highly rate academic or 
support services; and, students receiving support services were not expected to highly rate academic or 
vocational services. Table 4-12 presents the results of this analysis. 

Table 4-12 



7-12 STUDENT EVALUATION OF SERVICES BY DELIVERY SYSTEM 

(N « 1,627) 


Type of Delivery System 


Academic 
Services 


Vocational 
Services 


Support 
Services 


TOTAL 
SERVICES 


• Alternative schools (n=344) 


*2.97 


2.37 


2.47 


•2.55 


• Schools-within-schools (n=253) 


•2.65 


2.29 


2.43 


2.42 


• Classes/Labs/ Activities 

• Academic (n=380) 

• Vocational (n=241) 

• Support (n=309) 


*2.73 
*2.75 
*2.61 


2.22 
2.48 
2.27 


2.40 
2.38 
2.49 


2.41 
•2.52 
2.42 


• School-wide reform (n=100) 


•3.00 


*2.54 


2.42 


•2.61 


Scale: 1 .0 « Don't know/Didn't get help; 2.0 = Did not help at all; 3.0 = Helped a little; 4.0 
* = over 2.5 indicating some help 


= Helped a lot 



Table 4-12 illustrates several points. Among alternative school, school-within-school, and school- 
wide reform student participants, there was not equal satisfaction with all types of services. In fact, only 
academic support was perceived as having helped substantially. On the other hand, alternative school and 
school-wide reform students were apparently satisfied, overall, that services provided had been helpful. 

Comparing ratings among the three types of class/lab/activ'ty participants shows that vocational 
components received the highest ratings from students in both academic and vocational services; support- 
oriented components did receive higher ratings regarding social support than academic and vocational 
components. 

Overall student results suggest that alternative schools, vocational components, and school-wide 
reform efforts are "most effective " in delivering student services. 
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Further information on student services was obtained through teacher survey open-ended 
comments and interview data from students, parents and staff. Open-ended comments and interview 
results are summarized below with respect to each type of student service. 

Delivery systems and academic services: As depicted in Figure 4-1 (p. 74), the majority of open- 
ended teacher comments regarding program strengths were in this area. Regarding specific academic 
services, one-on-one tutoring and computer-assisted instruction were the most frequently noted program 
strengths, followed by self-paced instruction and small group instruction. Other comments suggested that 
student achievement had increased as a result of academic services and expressed satisfaction with specific 
delivery systems (e.g., alternative schools; extended-day programs). Additional comments indicated that 
the at-risk programs had allowed for schools to expand their range of offerings. Almost all types of 
services (e.g., CAI) were targeted for change by a few respondents. Most changes, however, centered 
around the need to "change" the curriculum within at-risk programs and set "higher standards' 1 for at-risk 
participants. Also noted were needs for more alternative educational services "outside" the regular school 
day. 

Interview responses from students, parents and staff varied widely-perhaps because of the diverse 
nature of individual at-risk programs and the isolation of discrete components. Nevertheless, some 
commonalities did occur. Respondents from all three regions most frequently agreed that students in at- 
risk programs had benefitted from extra individual attention which had been provided by a caring staff. 
The extra attention, respondents said, raised students' self-esteem and helped them improve their grades 
and stay in school. 

• Said one urban student: "Seeing teachers who care makes you want to graduate and 
want to come to school. Before, I wouldn't come to school. Now I try my best. I missed 
the bus once and walked to school because I wanted to come so bad." A reservation 
student added: "Teachers take ; ; me to talk to you about school." 

In a similar vein, tutorial programs drew praise from some respondents because they, too, 
provided extra individual attention. Tutoring provided a "support system" for one pupil, according to his 
parent. And yet concerns about tutorials were also expressed: sessions were too short; tutors needed better 
communication with students' teachers regarding homework assignments; and, tutorials weren't reaching 
the most at-risk students. After-school tutorials, in particular, were generally considered ineffective. 
Although one teacher said that her students had improved in math as a result of after-school tutorials, 
most respondents from each region said these programs suffered because they made the day too long for 
students, were poorly attended, and interfered with extra-curricular activities. 

Remedial computer labs received a modest endorsement from students for helping them in certain 
areas such as math and English. Also, a rer srvation school's move to fewer, longer class periods was 
widely supported by both stw uents and teachers at that school. 

Favorable opinions were expressed toward alternative programs, particularly by urban 
respondents. Students, parents, and staff agreed that alternative programs had prevented and retrieved 
many drop-outs, had provided a second chance for students to "grow up," and allowed a more flexible 
teaching style that was better suited for at-risk kids. 

• "I was thinking about dropping out as soon as I could," said an urban student. "[The 
alternative program] helped me to think about things." An urban parent added, "The 
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program has done, wonders (for my daughter). Her grades have come up. She's been able 
to make up almost all her credits." 

Students said they learned faster in alternative programs. They liked the curriculum and, as one 
said, they "were no longer embarrassed to ask questions" in class. They especially liked self-paced 
instructional programs. Students repeatedly cited the chance to work at their own speed in classes, 
whether it be f?st or slow, as one factors that helped them stay in school. 

• One reason self-pacing is effective, said an urban alternative school student, is because: 
"I work quickly and get frustrated" in a regular classroom. Said another: "I got bored 
with high school and I quit going even though I was getting good grades." 

Also noted in teachers' open-ended comments, curriculum concerns were apparent among staff 
interviewed as well. While several students and teachers defended the academics of alternative classes, 
characterizing them as interesting and rigorous, others said that credit requirements were too "easy" and 
that students weren't "intellectually challenged." Some called for a curriculum audit of their programs 
to help them shore up what they perceived as weak academics. Some alternative programs were also 
criticized by respondents for having poor facilities and insufficient books and materials. "Much of our 
stuff is outdated," said an urban alternative school teacher. 

Vocational services: Figure 4-1 shows that these services received the least number of comments 
regarding strengths. Of the strengths noted, however, a majority centered on the positive aspects of 
employment skills training and work experience programs. Other comments noted the benefits of 
vocational/occupational course work, career awareness opportunities, and "high tech labs." By far, most 
recommendations for change focused on the need to expand vocational/occupational course work. A 
number of instructors commented that "vocational training is inadequate." 

Interviews revealed that occupational training classes (such as computer skills, mechanical skills, 
silversmithing, and printing) received nearly universal praise from respondents who were involved with 
them. Students credited the classes with raising their self-esteem, increasing their motivation to attend 
school, and improving their sense of responsibility. 

• According to one reservation student, an applied technology class "makes things more 
interesting than in other classes." Another emphasized, "It is important to teach kids how 
to operate things." A teacher added, "The kids prefer to learn by doing." 

Classes that taught employability skills (such as interviewing, presentation, and punctuality) were 
also valued by students. The only negative comments regarding the classes were that they were too 
limited in scope or availability. 

Job placement programs for students, though primarily limited to a few urban districts, were 
strongly supported by those students, teachers, and employers who were involved with them. Students 
said they appreciated the chance to "get their foot in the door" witn employers and earn some money. 
Staff said the programs established valuable links to the community. And employers said the student 
workers were reliable and able to follow directions. 

• Said one student: "I learned how to sell merchandise." An employer commented "He 
outperformed a college student." Another employer added, "It reduced my employee 
turnover rate... and this is directly related to this program." 
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Across all regions and focus groups, respondents called for more, and better, practical programs 
geared toward teaching students employability skills and finding them jobs. 

Social /support services: Two of these services were singled out repeatedly in open-ended 
comments as program strengths: counseling services and social service linkages and referrals. Not 
surprisingly, these two areas also were the top two services targeted for improvement. Recommendations 
largely focused on the need to provide more counseling and earlier intervention, as well as to expand and 
improve linkages with other community-based agencies. Many noted that such linkages are beneficial, 
but stated that "services provided are inadequate." 

These themes also arose during interviews. For example, respondents considered support 
personnel such as case managers, social workers, and counselors as key players in most effective at-risk 
programs at urban and reservation schools. Students, parents, and staff credited those support personnel 
with helping provide links with social service agencies as well as providing "safe" links between students 
and teachers, between students and parents, and between schools and families. 

• Said an urban student: "(The case managers) make me feel wanted, like I have a future 
to look forward to. They're like my second parents. I know I wouldn't be in school if 
it weren't for them." 

Not all support personnel, however, were held in equal esteem. A contract counseling service at 
one school was accused of being "overpriced and overrated," and was said to have not "bought into the 
school's plan." A counseling department in one school was characterized as "lacking energy." And even 
some respected counselors and social workers were considered ineffective because they were 
overburdened with cases and could not provide regular individual attention to students. 

Regarding drug and alcohol prevention programs, a few respondents indicated that at-risk 
funding had augmented such efforts at their schools. Some felt that more of these programs were needed 
for at-risk youth. 

• One such program caused a reservation student to comment that he had learned "how 
to have fun without drinking" and had changed his attitude toward school. The program 
"was worth it," he said. 

In ethnic rural and reservation districts, a few students said they appreciated newly introduced 
instruction in cultural values. "Cultural counseling" was 4 adjunct of some support programs for at-risk 
students. 

Several parents and staff commented on discipline and attendance policies in at-risk programs. 
Many parents and a few staff attributed student successes to better "control" in alternative schools and 
classrooms. Some respondents, however, accused alternative programs of instituting double standards. 
They felt that at-risk students were "mollycoddled" and allowed to break rules. 

Respondents, especially students, repeatedly mentioned positive "outcomes" from the at-risk 
programs. Among these were: increased self-esteem, improved behavior, better attitude, more sense of 
responsibility, improved grades, increased motivation to attend school, and fewer dropouts. A few also 
mentioned reduced incidence of drug and alcohol abuse. In addition, students at rural and reservation 
schools emphasized that they perceived more "opportunities" for them as a result of at-risk programs. 
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Parent Services 

Many 7-12 at-risk pilot programs had "adjunct" parent involvement activities. These were 
investigated to determine which strategies worked better than others for older pupils' parents. Using a 
4-point Likert scale, teachers were asked whether or not each of seven strategies "worked for involving 
parents of at-risk students in their child's education." 

In recognition of the fact that not all parent services were addressed specifically under the 
auspices of the at-risk programs, services were evaluated in the total school context. The rationale was 
that perceptions of "working" should be reflected regardless of whether or not the service was an 
"official" part of the program. Staff ratings of services are presented in Table 4-13 which indicates that 
staff rated social events and verbal communication as working better than other parent involvement 
strategies. Parent workshops were perceived as least effective for parents of 7-12 students. 

Table 4-13 



7-12 PARENT SERVICES EVALUATION (N - 980) 

Has worked very well (needs to be maintained as is): 4.0 

School social event* 

More verbal communication between the parent and teacher/echool 
Formal home visits 

More counseling/psychological service* 
More written material mailed/sent to the home 
Structured clataes for parents (e.g., GED, ESL) 
Parent workshops 

Has not worked at all (needs total re vision): 1.0 

Scale: 1.0 = Has no: worked at all; 2.0 = Has not worked well; 3.0 = Has worked well; 4.0 = Has worked very well 



Regionally, parent services were rated highest by urban staff and lowest by reservation staff (see 
Appendix B, Table B-8). Grade 7-8 and 7-12 staff rated parent services more effective than did 9-12 
staff. Survey results analyzed by staff role indicate that specialists tended to rate parent services as 
working either well or very well, while administrators tended to say they "didn't work well." Vocational 
instructors tended to view parent services as more effective than did their academic colleagues. Both 
program staff and non-program staff who were aware of at-risk services in their districts viewed parent 
services as working better than did unaware non-program staff. 

Students were not surveyed regarding the effectiveness of parent services; rather, they were asked 
to indicate whether or not their parents had helped them more, or had come to school activities more as 
result of their participation in the at-risk programs. These results will be discussed in the Program 
Outcomes section of this chapter. 

Staff open-ended comments regarding parent services are presented in Figure 4-2 (p. 75). Parent 
services were considered program strengths by 5 percent of those responding; these services were targeted 
for change by 12.5 percent of those responding. While some staff indicated that parent involvement was 
not the responsibility of 7-12 programs, most comments were supportive of increased efforts to involve 
parents. 
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2.76 
2.74 
2.72 
2.72 
2.62 
2.55 
2.32 



An analysis of interview data regarding parent involvement yields two general themes. First, some 
parents and teachers in all three regions felt that support services provided by schools were effective 
means for both helping families and connecting them to the education process. These services were 
provided by school social workers, counselors, and case managers. 

• Praising one case manager, an urban parent said: "He does his best. If I have a 
problem, I call him or he calls me." 

Second, most respondents felt that overall parent involvement was weak. Respondents split, 
however, on the relative importance of this circumstance. Some, for instance, said that schools should 
Lake action to increase contacts with parents and attract them to school activities. 

• In this regard, one staff member said: "Parent contacts are what make the kids 
successful. Parental involvement is the bread and butter and butter of this program. If we 
don't have it, we lose these kids." 

Others asserted that seeking parent involvement at the secondary level was a relatively poor use 
of time and money. If parents weren't involved by the time their children were in high school, these 
respondents argued, then they probably never would be. Also, some respondents reported that many high 
school at-risk students neither needed nor wanted parent involvement because they no longer respected 
their parents. 

Respondents mentioned a few site-specific strategies that they considered effective in garnering 
parent involvement. At some urban schools, respondents agreed that parent workshops and special 
awards nights had successfully drawn parents to school. 

• Regarding an awards banquet in which he received recognition, one urban student said: 
"It was good to have my parents there because they are used to having me get in trouble. 
Since the banquet, they have helped me a lot with homework and helped me out more 
in general." 

Some effective parent involvement strategies that parents said schools should implement included 
home visits and frequent informational contacts such as notes or calls from teachers. Parents also said that 
schools should present a "welcome" atmosphere to parents, and that staff should exhibit an attitude of 
responsiveness to parental concerns. 

Staff Services 

As was done for K-3 at-risk programs, 7-12 evaluation efforts were directed toward determining 
satisfaction with staff development and training. Teacher survey respondents were asked to reflect on 
what activities "worked for training staff to work effectively with at-risk students." Table 4-14 presents 
the results of this evaluation and shows that all staff training activities included on the survey were rated 
below a 3.0, indicating that none were perceived as "working well." Conferences/academies and formal 
instruction (e.g., college classes) received the highest ratings; visits to other schools/programs were 
viewed as least effective. 
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Table 4-14 



7-11 STAFF SERVICES EVALUATION (N ■ 980) 

lias worked very well (needs to be maintained as Is): 4.0 



Conference*/ academic* 2.68 

Formal imtniction through a college course or other training claaa 2. 65 

Regularly scheduled program planning/development meeting! (i.e., specific to at-risk) 2.63 

Workihopa/In-service delivered by outaide consultant/trainer 2.58 

Workshops/In-service delivered by district staff 2.58 

Workshops/In-service provided on a regularly scheduled basis 2.54 

School/program visits and/or observations outside of own school 2,51 



Has not worked at all (needs total revision): 1.0 

Scale: 1.0 ■ Has not worked at all; 2.0 ■ Has not worked well; 3.0 = Has worked well; 4.0 ■ Has worked very well 



Urban staff tended to rate staff services higher than rural and reservation staff (see Appendix B, 
Table B-8). Once again, 7-8 and 7-12 staff rated services more positively than their 9-12 colleagues. 
Academic instructors, again, rated services lower than all other respondents by role, and program staff 
rated services higher than did their aware and unaware non-program colleagues. 

Figure 4-2 (p. 75) shows that of the 568 open-ended comments regarding program strengths, 42.2 
percent centered on the dedication and qualifications of program staff. In contrast, only 12.5 percent of 
the recommendations for change concerned staff. Of the recommendations made, however, respondents 
emphasized the need for additional staff training. 

With regard to interview data, teachers in some districts said they were satisfied with the number 
of opportunities available to them; however, most teachers called for more staff development 
opportunities in general. Suggestions from staff included the need for more on-campus workshops, more 
workshops dealing with native cultures, and more visitations and observations of other schools in order 
to stimulate new ideas. The most frequently recorded comment addressed the need for more conferences, 
workshops, and in-services aimed specifically at strategies for working with at-risk youth. 

PROGRAM OUTCOMES 

Outcomes for 7-12 at-risk programs were examined in several ways. First, during FY 1990-91, 
pilot site personnel reported the end-of-semester status of their student participants. Second, staff and 
students were asked their perceptions regarding program outcomes. Third, student outcome data- 
including absenteeism, credits earned, and achievement test data-were collected via the cohort study. 
Fourth, efforts were made to collect site-specific outcome data from participating districts through self- 
reports. 

Program Student Status 

Staff of the 7-12 programs were asked to record the end-of-semester status of their student 
participants with rspect to specific program outcomes (e.g., graduated, dropped out). Data were 
averaged for fall a^ spring semesters and are reported in Table 4-15. 
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Table 4-15 



7-12 PROGRAM PARTICIPATION OUTCOMES (N ■ 7,296*) 


POSITIVE OUTCOMES 


NEGATIVE OUTCOMES 


STATUS UNKNOWN 


Includes: 

• Remained in program 

• Traniition (regular program) 

• Graduated from HS/G.E.D. 

TOTAL 6,477 88.8% 


Includes: 

• Dropped out 

• Detention Center/Corrections 
TOTAL 198 2.7% 


Includes: 

• Transferred out-of-district 

• Don't know 

• Other (e.g., deceased) 

TOTAL 621 8.5% 


* Numbers reflect yearly averages; Pima County Detention Center ii excluded from this analysis. 



Table 4-15 shows that less than 3 percent of the total student participants were known to have 
dropped out; of the 621 students who transferred or whose whereabouts were unknown, however, there 
might be an additional percentage of dropouts. Even so, Table 4-15 indicates that a vast majority of 
student participants stayed in school or graduated. According to district reports, 62 students received a 
high school diploma or a G.E.D. as a result of program participation. Overall, positive outcomes reflect 
impressive numbers-particularly considering the fact that a minimum 5 percent of these students were 
"retrieved" dropouts upon entry into the at-risk programs 19 . 

Perceptions of Program Outcomes 

Staff Perceptions: Seven program outcomes 20 were included as items to be rated on the 7-12 
Teacher Survey. Two outcomes pertained to student outcomes, one to parent involvement, one to staff 
development, and three related to "overall program" outcomes. Table 4-16 shows that the mean score of 
all items is below the score indicating "achieved to a high degree." All but one outcome-improved 
student self-esteem-were perceived to have been achieved only to a very low to low degree (falling below 
a mean score of 2.5). 

Regional analyses indicate that rural staff were slightly more positive in their assessment of 
program outcomes than urban staff; both were notably more positive than reservation staff who perceived 
all outcomes as having been achieved to a very low to low degree. Again, 7-8 and 7-12 staff rated 
outcomes higher than their 9-12 counterparts. Administrators, specialists and vocational instructors were 
all more positive about outcomes than academic instructors. Program staff rated outcomes higher than 
did either aware or unaware non-program staff. 



The five percent estimate is known to be underestimated as not all districts reported this information. 

These seven program outcomes were gleaned from reviews of pilot program proposals and represent commonly shared or 
globally desirable outcomes for all programs. 
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Table 4*16 



7-12 STAFF PROGRAM OUTCOMES EVALUATION (N * 980) 



Type of Outcome 

Student 

Student 
Program in General 
Program in General 
Program in General 
Staff 

Parent 



Outcome 

Improved self-esteem of at-riik student! 

Improved academic achievement of at-risk students 

A comprehensive educational program 

Better linkages among school programs 

Better linkages with community-based organizations 

Increased staff skills for working w/at-risk students 

Improved parent involvement for parents of at-risk 



Very high degree: 4.0 

2.59 
2.42 
2.31 
2.28 
2.25 
2.16 
2.08 

Very low degree: 1.0 



Scale: Achieved to... 1.0 = Veiy low degree; 2.0 = Low degree; 3.0 = High degree; 4.0 = Veiy high degree 



Teacher survey respondents also were asked to reflect on the success of the 7-12 at-risk programs 
to provide services to "dropouts and potential dropouts" as specified by H.B. 2217 (1988). Specifically 
they were asked: "Do you feel that your district/school efforts to help at-risk pupils are, indeed, serving 
'dropouts' and 'potential dropouts' such that the students will complete high school graduation 
requirements?" Table 4-17 summarizes teacher responses to this question by region 21 . 

Table 4-17 reveals that one-third of the staff members in rural districts felt strongly that they were 
promoting completion of high school requirements; urban staff had the largest relative percentage of their 
respondents claiming programs were not keeping students on track. Broken down by ^de level, faculty 
working with higher grades (9-12) responded more strongly than their 7-8 colleagues that they were 
keeping students on-track. 



Table 4-17 



ARE 7-12 STUDENTS "ON-TRACK?" (N « 843) 


REGION 


Urban/ Suburban 
(n=202) 


Rural 

(n=565) 


Reservation 

(n«76) 


TOTAL 


YES, to a Large degree 


18% 


33% 


12% 


27% 


YES, to tome degree 


63% 


52% 


77% 


57% 


NO 


16% 


11% 


9% 


12% 


No response 


3% 


4% 


2% 


4% 



While Table 4-17 shows that a majority of respondents said they do believe that they are keeping 
students "on track," the diversity of opinion regarding this issue is reflected in the comments below: 



21 

Open-ended comments could not be linked with individual respondenti; therefore, analyses could not be conducted by 
respondent role and level of awareness. 

88 

ERIC IH) 



• YES, to a large degree: 



"Most of our graduates ...either have children or need to work to support families. 
Without our program and its flexible hours, these students would not have been able to 
complete their high school requirements and earn their diplomas." 

"The programs that have been initiated have led to a reduction of the drop-out [and] 
pregnancy rate. There is also little evidence of widespread drug abuse on campus." 

• YES, to some degree: 

J feel the students will probably graduate but with minimal skills. They seem to be low 
especially in the areas of reading/writing or communication skills..." 

"The at-risk program is good, but some students may use the program as an easy out in 
lieu of going to high school." 

• NO: 

"I don't believe that individualized/group instruction in [the at-risk program] meets the 
academic levels to make these students competitive for further academic/career training. 
I don't believe students are held accountable for misbehavior. Rules are not enforced, 
making the teaching situation for teachers very difficult." 

"Students will not complete high school because there are no basic training here 
regarding study habits that are needed at the high school level [sic]." 

While there nas been much faculty support for alternative programming for dropouts and 
potential dropouts, a concern expressed by many was capsulized in the following question: 
"While districts such as yours are implementing alternative programs/activities, some concern 
has been voiced that such programs often 'water down' curricula and allow students to exit high 
school without appropriate skills. Do you feel that this is occurring in your district/school?" Table 
4-18 summarizes the responses to this question by region. 

Table 4-18 



"ARE 7-12 STUDENTS RECEIVING WATERED DOWN CURRICULA?" (N - 986) 


REGION 


Urban/Suburban 

(n=202) 


Rural 
(n*-565) 


Reservation 

(n=76) 


TOTAL 


YES, to a large degree 


13% 


13% 


16% 


13% 


YES, to tome degree 


44% 


43% 


49% 


44% 


NO 


34% 


36% 


28% 


35% 


No response 


9% 


8% 


7% 


8% 



Table 4-18 reveals that many respondents across regions felt their curriculum was being 
"watered down" to some degree. Reservation staff appeared most concerned that this is an issue 
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(with only 28 percent "No" responses). The diversity of opinions on this topic is illustrated 
below. 

• YES, to a large degree: 

"At the high school many courses have been designated LEP etc. and are supposed to 
present the same curricula as their 'parent' courses. This does not happen and these 
students are not receiving equal skills. These classes end up as 'bozo' classes...." 

"The largest group of dropouts are those who cannot speak English. English is not 
required early enough, long enough, or with sufficiently high standards to allow the 
students to study a normal high school program. The 'watering down' is due to the same 
lack of English ability." 

• YES, to some degree: 

"Although we try to maintain high standards anu avoid 'watering down' the curricula, 
it does happen to some extent. Reasons for this stem from inadequate time available to 
work one-on-one with students. As a result, students much do much of their work on an 
individualized basis and therefore do not learn the content at as high a degree...." 

"I feel this is occurring... across the county with at-risk and all other students." 

• NO: 

"We have very competent and devoted teachers overseeing our program. Vocational and 
academic excellence is demanded and expected from our program...." 

"The same is expected of the at-risk student as the general populace. " 



In summary, the 7-12 Teacher Survey assessed staff perceptions of program outcomes. Sun ay 
results (Table 4-16) indicate that program outcomes were perceived as having been achieved only to very 
low degrees. While a majority of respondents viewed programs as keeping students on-track toward 
graduation (Table 4-17), many also felt that graduating students might have received "watered down" 
curricula (Table 4-18). 

Student Perceptions: Fourteen program outcomes were included as items to be rated on the 7-12 
Student Survey. Two outcomes each pertained to parent involvement and program staff. Ten pertained 
to changed attitudes and behaviors relevant to enhanced academic, vocational, and support services. Table 
4-19 depicts all students' ratings of program outcomes 22 . 

Only two outcomes were not viewed positively (means below 2.5): at-risk programs were not 
viewed as being the main reason for being in school or as providing opportunities for parent participation 
in school activities. Notably, perceptions of staff involvement were among the most highly rated 
outcomes; perceptions of parent involvement among the lowest. Student outcome data were analyzed by 
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With the exception of Pima County Hetention Center students, who were not administered a student outcomes section on 
their surveys. Pima County Detention Center is excluded from all remaining analyses and discussion of student outcome* - thus 
the total number of program components evaluated is 54. 
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region, grade level, and type of delivery system using chi square analyses and comparisons of frequencies 
and means. Significant chi square results were obtained for every analysis, suggesting differences between 
students on each variable. These analyses are summarized briefly. 

Table 4-19 



7-12 STUDENT PROGRAM OUTCOMES EVALUATION (N = 1,627) 


Type of Outcome 


The [at-risk program*] has... 


Agree a lot: 4.0 


Program 


• helped me to improve my grades 


2.96 


Staff 


S*i* t 111 t A 

• staff who really do care about me 


2.94 


Program 


• made me committed to graduating 


2.88 


Staff 


• more helpful teachers than others 


2.84 


Program 


• helped me set goals for my future 


2.82 


Program 


• improved my attitude about school 


2.77 


Program 


* let me cam school credits 


2.73 


Program 


• helped me feel better about myself 


2.72 


Program 


• prepared me for a "real world" job 


2.66 


Program 


• helped me better deal with problems 


2.58 


Program 


• made me come to school more often 


2.56 


Parent 


• gotten my parents to help w/school 


2.51 


Program 


* been the main reason I am in school 


2.41 


Parent 


• prompted parent participation 


2.17 






Disagree a lot: 1.0 


SCALE: 1.0 = Disagree a lot; 2.0 = Disagree a little; 3.0 = Agree a little; 4.0 = Agree a lot 

* Surveys were customized by name for each of the 54 programs or piogram components representative of the 7-12 at-risk 
pilot project; iter, wording has been modified slightly for inclusion in this table. 



Regarding parent involvement, urban programs appeared most successful in soliciting parents 
to come to school and in prompting them to help their at-risk teens Students in eight of 17 components 

I (47.1 percent of the urban programs) agreed that both parent outcomes were achieved. In comparison, 

three of 23 rutal components (13 percent) and none of the reservation components were "successful" in 
recruiting both parent support and participation. Across regions, students agreed that parents helped more 

| with their schoolwork as a result of programs in far more cases (27 of 54 programs or 50 percent) than 

they agreed that programs had provided their parents with opportunities to become involved (12 of 54 

m programs or 24 percent). 

Students in 7-8 grade components agreed with both parent outcomes more than did their 9-12 

■ peers. Five of 14 (36 percent) 7-8 program components involved parents compared to six of 40 (15 
* percent) at the high school level. Notably, all three of the rural programs cited above as being 

"successful" se*ve grades 7-8. None of the six delivery syp^ms was "successful" with respect to both 
I parent outcomes, and only three were regarded as having p .omoted parent support at home- -alternative 

schools (mean = 2.55), academic classes (mean = 2 51), and social/support activities (mean = 2.60). 
Parent participation mean ratings were in the negative range (i.e., 1.0-2.5) for all six delivery systems. 

With respect to staff involvement, urban programs appeared most positively perceived in terms 
m of caring and helpful staff. Students in 14 of 17 components (82 percent of the urban programs) agreed 

■ with both the fact that staff cared about them and that program staff were more helptul than other 
teachers. Reservation students in ten of 14 components (71 percent) also agreed that staff were helpful 
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and caring, while rural students in 14 of 23 components (61 percent) agreed. Notably, of the nine rural 
components where staff were perceived as either unhelpful or uncaring, seven components were located 
in two districts. Students in 7-8 grade components appeared much less positive regarding staff than 
students in 9-12 components. Eight of 14 (57 percent) 7-8 program components were perceived as having 
unhelpful and/or uncaring staff compared to 12 of 40 (30 percent) at the high school level. 

Delivery system differences were most pronounced with respect to alternative school students, 
whose mean ratings regarding staff helpfulness and caring are the only means above 3.0 (3.2 and 3.15 
respectively). School-within-school and school-wide reform teachers were perceived as next most helpful 
(means = 2.87). Discrete components' staff, although generally perceived positively, were not considered 
as helpful as staff in other delivery systems. All delivery systems were perceived f n have caring staff. 
Excluding alternative schools (which have already been addressed), staff were perceived as most caring 
in the following order: social /support components, academic/instructional components, vocational 
components, schools-within-schools, and school-wide reform efforts. 

Regarding program outcomes, rural students were most strongiy positive regarding overall 
outcomes; however, roughly only one-third of both the rural and urban programs were rated by students 
as having produced all 10 outcomes. Across regions, students credited programs with improving their 
grades, making them committed to staying in school, and helping them set future goals. Urban students 
also credited programs with improving their self-esteem; rural students credited their programs with 
helping them improve their attitude toward schooling; reservation students credited their programs with 
preparing them better for jobs in the "rail" world. Perhaps not surprising, high school participants agreed 
more with program outcome statements than did junior high students. 

Student outcome ratings by delivery system are presented in Table 4-20. For heuristic reasons, 
outcomes were conceptualized and categorized to correspond to primary academic, vocational, and 
support service outcomes, as well as to more general outcomes. Table 4-20 shows that students in 
alternative schools rated the primary academic outcomes highest; students in academic components rated 
them second highest. The highest means regarding primary vocational outcomes came from students in 
vocationally-oriented components; likewise the highest support services ratings came from students in 
support service components. For program outcomes in general, alternative school students consistently 
rate all outcomes higher than students in the other delivery systems. 
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Table 4-20 



STUDENT RATINGS OF PROGRAM OITCOMES P V DELIVERY SYSTEM 



Primary Academic 
Outcomes 



Grad - 
improved 



Earned 
more 
credits 



Primary Vocational 
Outcomes 



Better 
prepared 
for a job 



Help 
setting 
goals 



Primary SupjMit 
Services 
Outcomes 



Better 
self- 
esteem 



Better 
coping 
slrilLi 



Program Outcomes in General 



Stayed 
in school 



More 
committed 
to school 



Attend 
more 
often 



better 
attitude 
re: sen. 



3.23 
2.94 
3.08 
2.62 
2.80 
3.03 



3.18 
2.81 
2.72 
-2.40 
-2.48 
2.95 



2.77 
2.63 
-2.49 
2.89 
2.65 
2.57 



2.90 
2.75 
2.69 
3.01 
2.79 
2.86 



2.86 
2.57 
2.57 
2.76 
2.88 
2.58 



2.74 
2.51 
-2.42 
-2.44 
2.81 
2.55 



2.83 
2.65 
-2.23 
-2.20 
-2.31 
-2.12 



3.24 
2.88 
2.69 
2.73 
2.93 
2.80 



3.02 
2.63 
-2.35 
-2.35 
2.52 
-2.44 



2.95 
2.76 
2.64 
2.62 
2.78 
2.74 



Alt « alternative school; SWS = school-within-school; CL.A = classes/labs/activities where A = academic, V = 
vocational, and S = social/support; Reform = school-wide reform; means noted with a minus sign are generally negatively 
perceived with a majority of students disagreeing that the outcome was achieved. 



In sum, alternative school students clearly attributed changed attitudes and behaviors to their 
participation in these programs. Notably, SWS students were the only other gioup of students to perceive 
all outcomes positively. Vocational and support components were perceived positively by those directly 
participating in them. Support service components, in particular, appeared to be have a positive effect 
on attitudes. 

Student Attrition: Can programs make a difference if the students don't remain in school? 

As was done for K-3 programs, student attrition was analyzed for 7-12 programs. An original 
"cohort" of 1,307 7-12 students was identified in FY 1989-90. Table 4-21 presents the status of these 
students (by region) as of the end of FY 1990-91. 



Table 4-21 



STATUS OF 7-12 COHORT (End of FY 1990-91) 


REGION 


Original 
Cohort 


Graduated 


Dropped 
Out/DOJC 


Unknown* 


Promoted 
(Remaining) 


Urban 
Rural 

Reservation 
TOTAL 


431 
599 

277 

1,307 


62 14% 
55 9% 
14 5% 

131 10% 


67 16% 
79 13% 
52 19% 

198 15% 


210 49% 
339 57% 
109 39% 

658 50% 


92 21% 
126 21% 
102 37% 

320 25% 


♦Unknown includes students "lost" due to missing data and students transferring out of the original dii 


rtricts. 
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Table 4-21 shows that of the original cohort, only 320 pupils (25 percent) of the original cohort 
are still available for longitudinal tracking. Over the course of two years, 10 percent of the cohort 
graduated, 14 percent dropped out, 1 percent were incarcerated, and 50 percent were unable to be tracked 
(12 percent transferred and 38 percent were "lost" to missing data). Urban areas reflected the highest 
graduation rates, while reservation areas had the highest reported dropout rates, and rural areas had the 
highest number of students recorded as having transferred (making the percentage of "unknown" in this 
region particularly high). As of the end of FY 1990-91, the urban and rural cohort encompassed only 21 
percent of the original students; the reservation cohort retained 37 percent of its original composition. 

When these data are compared to outcomes reported in Table 4-15 (p. 87), an apparent 
discrepancy is revealed between data sets. However, removing students "lost" through missing data from 
Table 4-15 and recalculating outconas on the remainder of the cohort students produces the following- 
there are 56 percent "positive" outcomes (remaining in school/graduated), 25 percent "negative" 
outcomes, and 19 percent for whom outcomes are unknown. Comparing data from Tables 4-15 and 4-21 
shows that single school-year reports (as reflected in Table 4-15) reflect higher percentages of positive 
outcomes/lower percentages of negative outcomes than do multi-year reports. More than anything else 
however, the comparison suggests that a majority of "negative" outcomes and "unknown" student statuses 
are attributable to events that occur during summer months which are refl ected exclusively on Table 4-21 . 

In sum, high rates of student attrition are apparent in the cohort. Assessing the nature of student 
attrition is difficult, however, because of the extent of students "lost" through unreported data. Their 
status is not altogether clear. What is clear is that very few of the students originally targeted for tracking 
can be tracked through FY 1991-92. 6 * 



Student Outcomes 



In an additional endeavor to gauge whether or not programs have made a difference in students' 
academic performance, three primary student outcomes-attendance, achievement, and credits earned- 
were examined for 7-12 cohort students for whom three years of data were available. 

7-12 Student Attendance: In the Arizona At-Risk Pilot Project FY 1989-90 Project Report 
(Bierlein et al., 1990), a two-year trend reflected an increase in absenteeism among 632 of the 7-12 pilot 
program students. For this report, three years of attendance data were available for 527 students. Overall, 
absenteeism has not continued to rise during the third year of implementation (i.e., students have attended 
more). In fact, FY 1990-91 attendance is the highest of all three years studied, recording the lowest 
number of days absent and the highest number of days enrolled 23 . 

Total and regional attendance patterns for the 7-12 cohort are depicted in Figure 4-3. Regional 
analyses show that the most dramatic change in attendance occurred among urban teens. Rural students' 
attendance remained relatively stable and absentee rates for the 57 reservation students increased steadily 



23 

Thcae are calculated by taking the average number of day. absent divided by the average number of dayi enrolled. 
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Figure 4-3 




7-12 Student Achievement: ITBS and/or TAP scores for students were examined to discern any 
changes in test performance over time. Changes in student performance, however, cannot be attributed 
solely to participation in the at-risk program. Three-year trends in student achievnent were examined 
using normal curve, equivalent (or NCE) scores. Figure 4-4 portrays NCE scores for reading, language, 
and math subtests of the ITBS/TAP, by total and by region. 

These data show t' c the over 260 students for whom three years of data were available, 
reading scores have remained relatively stable for the total population, and for each region. All FY 1990- 
91 scores are within 1 percent of 1989-90 scores, and show overall gains since FY 1988-89. An overall 
trend toward increased language skills is depicted on Figure 4-4, which also shows steady language gains 
in rural and reservation areas. Math scores, too, have remained relatively stable for the total population 
and for each region. FY 1990-91 total scores are slightly lower (i.e., _.5 percent) than 1989-90 scores; 
however, they are higher than FY 1988-89 scores. 

In summarizing Figure 4-4, students show gains in all three areas compared to year one (FY 
1988-89) NCE scores, wiVi most gains reflected in language. In a majority of cases, rural at-risk students 
out-performed their urban and reservation peers in all areas. Notably, reservation students demonstrated 
considerably lower reading scores than their peers. 
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Figure 4-4 



AVERAGE ITBS NORM REFERENCED TEST SCORES (7-12 COHORT <. 266*) 
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ITBS/TAP NCE scores were also examined in relation to grade equivalent (G.E.) scores. This 
analysis showed that students made steady developmental progress from grade-to-grade. However, the 
question was posed: M What kind of progress is being made?" Taole 4-22 shows the median and range for 
grade equivalent scores by grade level using FY 1990-91 data. Using the median (which indicates the 
grade equivalent at which SO percent of the students score below and above), a majority of students in 
general demonstrate skills "below" grade level. Range scores, however, revealed variations in individual 
skill levels showing that there are students performing at or above grade level. 

Table 4*22 



FY 1990-91 GRADE EQUIVALENT 1 


[TBS/TAP SCORES FOR 7-12 COHORT (N 


- 264) 




FY 1991 
Grade Levels* 


n 


Reading G.E. 
Median 


Range 


n 


Language G.E. 
Median Range 


n 


Math G.E. 
Median 


Range 


8th 


78 


6.8 


3.4-19.8 


78 


6.8 


4.1-19.5 


81 


7.2 


5.0-19.2 


9th 


74 


7.8 


3.0-14.7 


75 


7.9 


4.2-14.4 


76 


7.6 


3.6-13.2 


10th 


34 


7.6 


3.6-16.1 


34 


«.l 


4.0-14.0 


35 


7.8 


4.9-16.2 


1 1th 


42 


8.2 


3.7-17.1 


43 


9.9 


4.2-14.0 


43 


8.1 


4.8-14.0 


12th 


24 


10.3 


7.0-14.4 


24 


9.9 


6.0-13.5 


24 


8.7 


5.0-15.8 


* Each deiignated grade level represent a subgroup of the cohort for whom three yean of data were available. 
Therefore, 1991 eighth grade™ were 1988-89 sixth graders and 1990-91 seventh graders, and so on. 



An additional analysis of TAP scores considered the exit skills of high school seniors. Twelfth 
grade NCE results were compared for two consecutive years. The hypothesis was that changes in twelfth 
grade students' scores might reflect changes as a result of program participation (even though there are 
other variables which contribute to such changes). Figure 4-5 depicts trends in twelfth grade performance. 
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Figure 4-5 



ARE 7-12 STUDENTS "ON-TRACK?": TWELFTH GRADE STUDENTS' TAP PERFORMANCE 
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FY 1989-90 Twelfth Grade Student! (Reading n = 46; Language n « 40; Mtth n = 43; Composite n - 40) 
FY 1990-91 Twelfth Grade Studcntt (Reading n = 40; Language n = 41; Math n * 40; Composite n = 37) 



Figure 4-5 shows that on the composite, consecutive groups of seniors performed at similar 
levels. However, recent FY 1990-91 graduating seniors exited with higher level language and math skills, 
and lower reading skills, than their predecessors. 

7-12 Credits Earned: As explained in the Arizona At-Risk Pilot Project FY 1989-9C Project 
Report (Bierlein et al., 1990), 20 credits are required by the state for graduation, and therefore, five 
credits is the "average" number of credits needed per year for four years of high school. For all students 
for whom data were available for both FY 1989-90 and 90-91, a little over one-third of the students (34.6 
percent) earned five or more credits; two-thirds (63.6 percent) earned less than 5 credits. Analyses next 
focused on examining the percentage of students by grade who were "on-track" regarding credits earned. 
This analysis is presented in Table 4-23. 
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Table 4-23 



I 1 " ' '' =^^»n mi | ,_ ; 

9-12 CUMULATIVE CREDITS EARNED (as of FY 1990-91) 


Grade 


< 5 




5-9.5 


10-14.5 


15-19.5 


> 20 


% "on-track" 




credits 




credits 


credits 


credits 


credit* 




9th (n=185) 


49* 




4h% 


3% 


<1% 


<1% 


51% 


10th (n=138) 


1096 




46% 


35% 


7% 


2% 


44% 


llth(n=55) 


4% 




13% 


38% 


31% 


14% 


45% 


12th (n=68) 


0% 




3% 


12% 


15% 


71% 


71% 



Table 4-23 indicates that a little less than half of students grades 9-1 1 were "on-track" in 
accumulating an average of five credits annually, while 71 percent of grade 12 students accumulated 
enough credits to graduate. The shaded area in Table 4-23 indicates those students who were not on track, 
and might be considered "overage" going into the next grade. 

District Self-Reported Otcomes: Morrison Institute encouraged all districts to submit self- 
evaluation data, and eight of the 21 program sites (38 percent) took advantage of this opportunity. The 
following discussion is not a comprehensive report of all district self-evaluation efforts. Rather, it 
illustrates some of the types of self-evaluation being conducted. 

Case Study Hh This rural district offered both an alternative school for students in grades 7-12, 
and a school-within-school (SWS) intervention program aimed primarily at seventh and eighth grade 
students. Comprehensive district evaluations were conducted regarding student enrollment, credits earned, 
and student attrition. Follow-up studies were also completed on all students. Computer-assisted instruction 
was an integral part of the services provided, and a separate evaluation of student lab usage and outcomes 
was conducted. During FY 1990-91, special emphasis was placed on offering vocational services to SWS 
students and an evaluation was also completed of this component. 

Evaluation results indicate 213 (74 percent) of tne 287 "potential dropouts" enrolled in both 
programs remained in school and earned a combined total of 492 credits, averaging B to B+ grades. The 
evaluation of the computer lab revealed a significant increase in usage and numbers of credits earned- 
particularly in mathematics. In fact, pre- and posttest for students receiving CAI in math revealed an 
average grade level increase of 1.54. Regarding this outcome, the district concluded: "This is exciting 
in light of the fact that several of the students were having difficulty answering even the most basic 
problems... at the beginning of the year." In citing specific accomplishments of alternative education 
students in this program, it is noteworthy that: one of four valedictorians was an alternative education 
student; seven of J 8 honor students were in this program; 10 students received scholarships; one wai 
inducted into the National Honor Society; and 15 others received awards in other subject areas, including 
one State Writing Contest winner. 

Case Study #2: This urban district offered discrete components in each of the three areas targeted 
for intervention: academic, vocational, and support. Although evaluation data were submitted in several 
areas, this case study focuses on the vocational aspect of the program. District results indicated that 
following career awareness instruction provided at five secondary schools, 21 students were subsequently 



99 

1 0 V 



trained and placed through the Work Experience Program. These students were evaluated through regular 
contact reports, student self-reports and employer evaluations. The district concluded: 

"It is evident... that the job placement component is viewed extremely positively by both the 
participating students and their employers. Students value the excellent preparation for the 
transition from school to the work place which increases their Motivation to stay in school and 
study harder. Employers are enthusiastic in regard to 1 1 job-related skills and attitudes, such as 
attendance, punctuality, initiative and positive attitudes toward work and fellow workers. * 

Case Study #3: This rural district examined changes in student self-esteem, and work-rslated 
behaviors and attitudes as a result of guidance counseling. Pre- and posttest results were available for 1 13 
students regarding self-esteem, and for 32 students regarding career attitudes. The district indicated that: 

"Significant growth occurred in almost all areas [and] demonstrate the effectiveness of the 
[program]. " 

Case Study H\ Unique among the project, the Pima County Detention Center documented its 
student population and outcomes perhaps more than any other at-risk pilot ?Ue. This program served 
1,063 students during FY 1990-91. To illustrate how the center functioned a "dropout interdiction" 
program, data were tabulated for students entering the detention center from two local school districts. 
The project director reported that: 

"...out of 341 students, 281 were retrieved [dropouts). This mean that after departing 
detention, 281 students followed one of the following tracks: 1) they reentered their home 
school (60 percent of all students in program will return to their home school) as a 
condition of their release; 2) they entered a court-related program or a community-based 
program that had an educational component in pLce; 3) they were released to Department 
of Corrections which has a continuing education program." 

Furthermore: 

Before [this program], these students were ignored educationally. Even worse, students 
who were enrolled in schools became dropouts after 10 days... [this program] has 
remedied this situation and has raised the consciousness of the juvenile court and made 
them more aware of their responsibility to address detained youths ' education needs. 



7-12 PROGRAM EVALUATION CONCLUSIONS 



PROGRAM IMPLEMENTATION CONCLUSIONS 

A comparison of survey ratings on implementation success/barriers, open-ended comments, 
interview data, and administrative turnover yields several recurrent concerns: 

• Poor/inadequate communication has adversely affected program implementation. 

• Coordinating/integrating programs has beer problematic; establishing school-community 
(X)llahoration is an issue still to be addressed, particularly for rural and reservation programs. 
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• Unreliable funding and inappropriate funding cycles have created difficulties implementing 
programs as planned. 

• Qualified and committed staff can either "make or break" a program; adequate and appropriate 
staff training is required to produce qualified staff and update existing staff. 

• Strong program leadership and administrative support are essential for program success; the 
lack of stable leadership in rural and reservation areas is, therefore, cause for concern. 

Although there are some concerns regarding program implementation, it is important to recap the 
following positive findings as well: 

• Programs, in general, have been implemented as planned; they served an estimated 9,385 
students, established communication with over 1,500 parents and offered specific services to 
many others, and provided staff training for personnel at all 21 program sites. 

• Of 15 factors identified as affecting program implementation, a majority (60 percent) were 
generally perceived by staff as contributing to successful implementation. 

• Responses to oj en-ended questions revealed more positive than negative comments regarding 
programs. 

• Interviewees generally felt that their programs were implemented successfully. 
PROGRAM SERVICES CONCLUSIONS 

Student Services 

Analyzing teacher and student survey ratings, open-ended comments, and interview data, the 
following conclusions are derived regarding student services: 

• Statistical and qualitative differences appeared among all surveyed 7-12 stiff when data were 
analyzed by region, grade level, role, and level of program awareness. Insofar as these 
differences indicate factions among the staff, there appears to be little concensus as to "what 
works" specifically for at-risk youth. 

• Alternative programs hold promise as a delivery system; however, to be accepted and 
"credible/ attention must be paid to the quality of the curriculum, program standards, and 
communication efforts to convey program quality to non-program staff. 

• Aggregate staff survey results suggest a preference for delivery systems offering "traditional" 
courses of study supplemented by discrete program components; in contrast, program staff and 
students prefer more holistic delivery systems. 

• There is strong consensus that for programs to be successful, staff must be committed to 
provid'ng individual attention and students must feel cared about. 

• Academic/instructional services are positively perceived across 7-12 at-risk programs; however, 
teachers and students do not agree on preferred instructional strategies. 
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• Students and program staff involved with vocationally-oriented activities perceive them quite 
positively. However, vocational services were limited in range and accessibility, especially at the 
7-8 grade levels. 

• Social/support program components and services are valuable elements of at-risk programs, but 
more services and additional qualified personnel are needed, as are more linkages with outside 
agencies. 

Parent Services 

Regarding parent services, several conclusions can be drawn: 

• Parent services are weak at the 7-12 level apparently because parent involvement was not an 
emphasis of 7-12 at-risk programs and because 7-12 staff held mixed opinions on the importance 
of parent involvement. 

• Of the parent involvement activities that were implemented, social events and efforts to reach 
parents individually are perceived as most effective. 

• Support services extending to families of at-rLl: youth (e.g., family counseling; home visits) 
hold promise as "effective" strategies for involving parents, but they need to be expanded. 

Staff Services 

Staff surveys, in conjunction with interview and external evaluation data, suggest the following 
conclusions regarding staff services: 

• Program staff, particularly in the urban areas, appear most satisfied with staff training 
opportunities. Better opportunities need to be made available for all program and non-program 
staff, particularly in the rural and reservation areas. 

• Staff need more training regarding specific strategies for working with at-risk youth. 

• Conferences and formal classes are preferred training methods; access to formal classes is 
limited and problematic for staff in isolated, rural, and reservation areas. 

PROGRAM OUTCOME CONCLUSIONS 

A comparison of teacher and student perceptions of program/student outcomes, student outcome 
data, and district-reported data, in conjunction with a study on student mobility, indicates the following: 

• School-year reports indicate that nearly 89 percent of the students participating in the 7-12 at- 
risk programs stayed in school or graduated; less than 3 percent were verified dropouts during 
FY 1990-91. Multi-year reports indicate that 56 percent of the students tracked over time stayed 
in school or graduated, while only IS percent were verified dropouts. 

• Student attrition, particularly during summer months, causes great difficulty in tracking at-risk 
students. This, in turn, clouds the issue of student transience versus dropouts at the upper grade 
levels. 
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• Staff and students agree that the greatest program impacts are: improving students' self-esteem 
and increasing their achievement. 

• Staff believe that they are keeping students "on-track" toward graduation, but some feel that 
they are simplifying the curriculum in order to do so. 

• Students in alternative schools are most positive about their program experiences; however, 
each type of program appears to be accomplishing its primary goals. 

• In FY 1990-9 1 , absenteeism decreased for the first time since the inception of at-risk programs. 
This implies that at-risk youth received "additional" instructional days. 

• For the teenagers represented by the 7-12 cohort: 1) ITBS/TAP NCE results show relatively 
stable performances in reading and math, and an overall trend toward increased language skills. 

• For the teenagers represented by the 7-12 cohort: 1) ITBS/TAP NCE results, when converted 
to grade equivalent scores, show that students are making steady developmental progress; 
however, a majority of students remain "below" grade level. 

• For consecutive groups of seniors represented by the 7-12 cohort: TAP NCE results show that 
FY 1990-91 students are exiting with higher language and math skills and lower reading skills 
than their predecessors, but overall performance has remained stable between years as reflected 
by the composite scores. 

• Results indicating net gains in language are encouraging, as these skills are correlated with 
academic achievement; low reading levels, however, are cause for concern-particularly among 
reservation youth. 

• A majority of at-risk students are not earning an average of 5 or more credits per year. 

• A majority of at-risk students in grades 9-1 1 are not "on track" regarding cumulative credits 
earned; however, 71 percent of the twelfth grade cohort students earned sufficient credits to 
graduate. 

• District-specific outcome data suggest positive impacts from programs and also illustrate the 
importance of locally-conducted evaluations in portraying specific program effectiveness. 
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Chapter 5 

AT-RISK PILOT PROTECT BUDGET BREAKDOWN 



During FY 1990-91, $5.5 million was available to support the K-3 pilot project while $2.2 
million was available for the 7-12 project. Specific grant awards ranged from $13,745 to $270,686 for 
the individual K-3 programs, and $75,000 to $256,425 for the 7-12 programs 24 . During 1991, S.B. 1079 
provided an additional transition year to the project, established a Joint Legislative Study Committee, and 
required an analysis of how program funds were expended. Specifically, the bill required a study to: 

"examine the amounts spent during fiscal year 1990-91 for various program activities... Any 
funding method that is determined for meeting the needs of preschool through twelfth grade at- 
risk pupils &all be designed to limit those costs that are associated with activities that are 
administrative and that do not provide direct services to pupils, parents or teachers." 

An independent budget study was therefore undertaken by Morrison Institute as an adjunct to the 
original evaluation study. The specific types of budget data to be collected, and the data collection format, 
were determined in collaboration with several pilot districts. The intent of the budget study was not to 
conduct a cost/benefit analysis, but rather to describe by category how funds were spent by pilot 
programs during FY 1990-91. Budget data were requested from all at-risk programs, with data analysis 
focusing on four key questions: 

1. How much of the at-risk funds were spent on direct services to students, parents, and teachers? 

2. What percentage of total district maintenance and operations budgets did FY 1990-91 at-risk 
funds account for? 

3. What was the average cost per pupil, and what were the ranges of cost per pupil, based on total 
at-risk funding expended? 

4. What is an appropriate level of funding to meet the needs of at-risk students? 



BUDGET ANALYSIS AND RESULTS 

This section highlights budget data results for K-3 and 7-12 programs. Data were examined in 
several ways (i.e., including and excluding carry forward monies; by phase; by size of district; by region; 
and as an aggregate). In examining information related to cost per pupil, it is important to remember that 
some variation exists between programs in reporting the number of students served. For example, 
programs that incorporated the use of classroom aides or developmentally appropriate practices typically 
counted all students as receiving those services, whether or not they were identified as at risk. Therefore, 
those districts showed a large number of students served and a relatively lower cost per pupil than 
programs reporting a more specifically targeted group of at-risk students. The methodology for and 
limitations of budget collection efforts are described in Chapter 2 and Appendix D of UJs report. 



24 

The actual amount of K-3 grant funding available during FY 1990-91 wai greater than these figures since most programs had 
carry forward monies from the previous year; the range listed for the 7-12 programs already included the previous year's carry 
forward funding. 
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K-3 PROGRAM BUDGETS 



An analysis of program expenditures for FY 1990-91 by function reveals that, of total funding 
amounts, 69 percent was spent on direct services to students, 2 percent on services to parents, and 4 
percent on services for staff; 8 percent went to administrative, evaluation, and indirect costs; and 16 
percent was carried forward to FY 1991-92. Data revealed that six of 22 Phase I programs (27 percent), 
and eight of 13 Phase II districts (62 percent), carried forward at least 20 percent of their hinds. 

When the 16 percent carry forward is excluded, the remaining funds represent total funds actually 
expended in FY 1991-92. As illustrated in Figure 5-1, 83 percent of funds were expended on direct 
student services, 4 percent on parent services, and 5 percent on staff services. The remaining 8 percent 
was spent on school and district level administration, evaluation, and indirect costs. Budget expenditures 
by individual program, with breakdown by function, are presented in Appendix C, Table C-l. 

Figure 5-1 



PERCENT BREAKDOWN OF ALL K-3 CATEGORIES 
(excluding carry forward to FY 1991-92) 



Ds jdent Services 83% 

I Parent Services 4% 

QStaff Service 5% 

iSchool Admin 2% 

I District Admin 3% 

D Indirect Costs 2% 
§ Program Evaluation 1% 

100% 



At-risk ftinding levels also were examined relative to each district's total maintenance and 
operations (M & 0) budget. Budgets from a sample of districts, representing a range of funding levels, 
district size, and regions, were calculated as a percent of total district M & 0 budgets 26 . Results of this 
analysis revealed that, for the vast majority of K-3 districts, the at-risk pilot funding represented only 



Forty of 42 districts were included in the data analysis: one district did not submit data in time to be included in the analysis, 
while one district failed to submit any budget data at all, 

26 Budget figures were obtained from "Statistical and Financial Data for Fiscal Year 1989-90," Arizona Department of 
Education, December 1990. This analysis used grant amount* for FY 1989-90 since state data on total district M & O figures were 
not yet available for FY 1990-91. 
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13% 




slightly more than 1 percent of their total M & 0 budget. In two small districts, however, at-risk funds 
accounted for 7 percent and 10 percent of the total M & 0 budget. 

Appendix C, Table C-2 contains a compilation, by region, of data gathered from districts 
including: 100th day average daily membership (ADM) for K-3 students; total number of students served 
as reported by districts; total budget for direct student services; cost per pupil based on direct student 
services expenditures; total program budget (excluding carry forward); cost per pupil based on total 
dollars expended. Based on expenditures for direct student services only, the average cost per pupil was 
$189. When total budget expenditures were used, the cost per pupil was $227, with a range from $76 
to $2840 per pupil. 

Phase I and phase II districts had similar costs per pupil. When expenditures were analyzed by 
region, however (see Figure 5-2 below), it was found that rural programs had the highest per pupil cost 
($305), followed by reservation programs ($233) and urban programs ($192). It is important to note, 
however, that while rural and urban areas had the same number of funded projects (14 each), rural pilot 
sites were smaller and therefore had far fewer students to serve (14 urban projects served 12,772 
students, while 14 rural projects served 5,328). Thus, a higher cost per pupil would be expected. When 
the data were analyzed by size of district (as determined by lOOth-day K-3 ADM for 1990-91), a similar 
pattern emerged. Small districts (ADM <. 500) spent an average of $449 per pupil; medium districts 
(ADM = 50i-1000) spent an average of $285 per pupil; large districts (ADM > 1000) spent $151 per 
pupil. 



Figure 5-2 



AVERAGE COST PER K-3 PUPIL 



BY REGION 



BY DISTRICT SIZE 




Programs that expended a large number of dollars per pupil did not necessarily offer a large 
number of services. In fact, the number of program services offered was about evenly distributed across 
programs regardless of cost per pupil expenditure. More important than the number of services offered 
is the intensity and relative effectiveness of those services. For example, a highly focused one-on-one 
tutoring program is more costly, but has a large achievement effect on only a small number of students. 
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By contrast, a strategy such as the implementation of developmental^ appropriate practices is much less 
costly on a per pupil basis because it affects a large number of students (i.e., all students in the classroom 
or the school). To summarize, strategies that focus individual attention on a small number of students are 
mere costly but possibly more effective. Examining the relative effectiveness of specific strategies will 
be a focus of the year four evaluation study. 

7-12 PROGRAM BUDGETS 

All 13 7-12 programs were includui in the budget analysis. Budget expenditures by individual 
program, with breakdown by function, are presented in Appendix C, Table C-3. With carry forward 
included, 84 percent of funds were expended on direct student, parent, and staff services. With carry 
forward excluded, 89 percent of funds were expended on these services, and the remaining 1 1 percent 
was allocated to administration, evaluation, and indirect costs, as shown in Figure 5-3 below. 

Figure 5-3 



PERCENT BREAKDOWN OF ALL 7-12 CATEGORIES 
(excluding carry forward to FY 1991-92) 



D Student Services 84% 

I Parent Services 3% 

Qstaff Services 2% 

ischool Admin 8% 

I District Admin 1% 

D Indirect Costs 1% 

BProgram Evaluation 1% 
100% 



S4% 




As with the K-3 programs, 7-12 funding was calculated as a percent of total district M & O 
budgets. The funding level for 7-12 districts was found to be slightly higher than for K-3 programs, with 
the majority of at-risk funds constituting between 2 and 3 percent of total M & 0 funds. 

The compilation of data on total number of students served, expenditures for direct student 
services, total expenditures, and costs per pupil for 7-12 programs is shown in Appendix C, Table C-4). 
Using expenditures for direct student services only, the average cost per pupil was $211. When total 
project expenditures were accounted for, the average cost per pupil was $251, with a range of $76 to 
$2648. Again, higher costs per pupil typically signified more intensified services for relatively fewer 
students rather than simply more services. Among 7-12 districts, for example, programs offering 
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alternative schools generally had a higher cost per pupil than those offering interventions within the 
regular education program that reached more students. 

The regional breakdown for 7-12 programs, shown below in Figure 5-4, reveals a different 
pattern from the K-3 programs. Urban districts showed the highest per pupil expenditure at $318, 
followed by rural districts at $235 and reservation districts at $214. The budget data for 7-12 were not 
analyzed by district size due to the diversity of grade levels being served (i.e., some served 7-12, some 
7-8 only, and some 9-12 only), and the diversity of programs offered (e.g., a county detention center; 
a consortium of nine districts). 



Figure 5-4 





DISCUSSION AND CONCLUSIONS 



The results show quite clearly that the vast majority of at-risk grant funds, both at the K-3 and 
the 7-12 levels, were expended on direct services to students, parents, and staff, with relatively small 
percentages allocated to administrative and indirect costs. Although there was a wide range from program 
to program in costs per pupil, the majority of programs clustered around the average cost per pupil ($225 
for K-3; $251 for 7-12). As would be expected, the strategies that required low student-staff ratios were 
more costly. Regional and size differences at the K-3 level presented an interesting perspective on 
"economies of scale." Programs in rural areas or in districts with low ADM's (often the same districts 
since rural districts tend to be small) had substantially higher per pupil costs, even though many small 
rural programs served their entire K-3 populations. Site evaluators and district staff reported that when 
these districts first developed their programs they were already behind urban districts in terms of 
availability and accessibility of resources (e.g., instructional materials, facilities, teacher training), thus 
adding to their per pupil costs. 

It is interesting to note the difference in per pupil spending patterns shown for 7-12 programs, 
especially the finding that urban districts had the highest cost. This may be explained by the fact that 
secondary schools tend to be about the same size regardless of their regional location. Therefore, the 
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factor of very small schools/districts that exists at the elementary level probably is not a factor at the 
secondary level. 

K-3 programs spent a surprisingly small percentage of monies (4 percent) on parent services, 
considering the wide range of services offered by most programs. It should not be inferred from this, 
however, that these services are not costly. Most parent services, actually, ■ e funded by other ^'strict 
sources. In addition, many parent services relied on the human resources already available in the Jets. 
For example, teachers made more home visits and more personal parent contacts as part of the normal 
job requirements, and without additional compensation. Whether duties such as making home visits should 
be an integral part of a teacher's job responsibilities is an issue that warrants further examination. Other 
services such as holding parent meetings and encouraging attendance at school events appeared to rely 
more on staff energy and ingenuity than on funding per se. 

The amount of funding carried forward to the next fiscal year, and the associated funding policies, 
require some scrutiny. K-3 funding was structured so that programs retained carry forward funds in their 
district budgets to spend the following fiscal year. One-third of all K-3 districts carried forward 20 
percent or more of their total funding amount from 1990-91 to 1991-92. Follow-up phone calls to project 
directors revealed that several districts used the previous year's carry forward to fund summer school or 
summer in-service activities held in July and August. In other cases, the funds were held in reserve for 
services to be offered the following year or simply were not expended by the end of the fiscal year. 

At the 7-12 level, however, no districts carried forward 20 percent or more of their funds, and 
the majority had no carry forward funds at all. This difference existed even though 7-12 districts were 
funded at a slightly higher level in terms of percent of their total district budgets. Under the 7-12 funding 
structure, however, districts were required to turn back monies not expended, and these funds were then 
re-allocated the next year. In other words, "if they didn't use it, they could lose it." 

When writing their initial grant proposals and funding requests, schools and/or districts had a 
great deal of discretion in defining their target populations (i.e., the total number of students they hoped 
to serve through the at-risk program). Theoretically, an "appropriate" level of funding per at-risk student 
could be derived from the average costs actually incurred by these programs, for if costs were 
significantly higher than the average expenditures, programs could have chosen to serve fewer students. 
If this were the case, an "appropriate" level for K-3 programs would be $227, while the amount would 
be $251 at the 7-12 level. However, other information has revealed that many of these programs 
(particularly the rural programs) had very limited resources available for at-risk programming prior to 
this grant. As a result, many programs attempted to provide at least some additional support to a larger 
group of students, not necessarily an "appropriate" amount of support. Plus many programs felt 
compelled to serve large numbers of students since evaluations in the past have frequently focused on 
quantity, not quality. Finally, interview data revealed that within many of the more "targeted" programs, 
funding limitations prevented them from being able to serve all students identified as needing these more 
comprehensive services. 

Moving beyond the context of Arizona's pilot programs, several recent at-risk initiatives within 
Kentucky offer a good point of reference for funding comparisons since this state's student population 
size and free/reduced students percentages are fairly similar to Arizona's. Kentucky's Educational Reforn, 
Act of 1990 added an at-risk funding weight of 0.15 per student eligible for the federal free lunch 
program ($328 per student during FY 1990/91). In addition, separate funding was provided to support 
Family Resource Centers for every school that has at least 20 percent of their students meeting the free 
lunch criteria. F»:.ding for these centers was based on $200 per eligible student, within a range of 
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$10,000 to $90,000 per center. In addition, separate funding is provided for teacher training ($25 per 
student). 

Using the Kentucky numbers as a guide, one can surmise that an "appropriate" funding level per 
at-risk student for Arizona would be higher than the average spent by the pilot at-risk programs. This is 
especially true if the goal is to not only address the academic concerns of at-risk students, but also to 
address the parental and social service issues. Evaluation results have illustrated that Arizona's pilot 
programs generally were able to focus on academic concerns, but most were unable to provide effective 
staff development activities and offered very little in the way of social service coordination. In Kentucky, 
oolicymakers choose to provide several separate funding streams to address these various issues. Arizona 
may wish to consider something similar, or should consider adopting an at-risk weight adequate to cover 
all aspects of at-risk programming 27 . 

Keeping in mind the four key questions posed at the beginning of this chapter, the following 
conclusions can be drawn regarding at-risk funding: 

• 92 percent of the at-risk funding at the K-3 level and 89 percent at the 7-12 level was expended 
on direct services to at-risk students, their parents, and the staff who teach them. Central office 
administrative costs represented 1 percent at the K-3 level and 2 percent for 7-12 programs-the 
remainder was used for school-level program coordination, evaluation, and indirect costs. 

• At-risk funding accounted for about 1 percent of the total district M & O budgets at the K-3 
level, and about 2 percent at the 7-12 level. 

• The average cost per pupil for K-3 programs was $227, with a range from $76 to $2840. The 
average cost per pupil for 7-12 programs was $251, with a range from $76 to $2648. The 
majority of programs at both levels clustered around the average. 

• An "appropriate" funding level per children should be no less than the average expended per 
student by these pilot programs. Dependent upon the expectations for training, parental 
involvement and social service coordination, the amount should be higher than the average (e.g., 
during FY 1990-91, Kentucky provided $528 per free lunch student-$328 in the formula for 
general programming and $200 for Family Resource Center support). 

• Given that it is more expensive for small/or rural uistricts and schools to pi^vide services to 
at-risk students, it is suggested that, in addition to a "base" weight, additional funding should be 
provided to small and rural schools, and for those programs that wish to develop more 
comprehensive services such as alternative schools. Further, a minimum amount (e.g., $5,000- 
$10,000) should be considered for all except the very smallest of schools. Without some 
minimum amount it becomes impossible to "purchase" additional personnel (and/or stipends for 
existing personnel) to provide the individualized support found effective with these at-risk 
students. 



It ihould be noted that recommendation* contained within the Governor's Task Force on Educational Reform (a* of October 
1991) have generally embraced ihc concept of aevcnal funding itrcams. This group has recommended the establishment of an at-riak 
funding weight, the provision of funding to establish a statewide parent training program, the creation of Family Resource Centers, 
and the provision of funding for teacher training. In addition, a variety of other at-risk initiatives an: also recommended (e.g.', 
preschool, prenatal, guaranteed scholarships). 



in 

lis 



Chapter 6 



SUMMARY AND RECOMMENDATIONS 



This third annual report has identified: 1) the general types of practices that hold promise for 
meeting the needs of at-risk students, and 2) the implementation processes that allow such practices to 
flourish. Thus far, the report has presented an extensive and detailed analysis of data related to the 
evaluation of the Arizona At-Risk Pilot Project. Specific strategies implemented within particular programs 
will be examined in depth during FY 1991-92 and reported in a "what works" document at the conclusion 
of the four-year study. 

This chapter synthesizes evaluation results with current educational research and presents the most 
consequential implications for developing policies and long-range plans affecting at-risk students. 
Information gathered during meetings with K-3 and 7-12 project directors in September 1991 is 
incorporated. During these meetings, formal discussions focused on generating solutions to identified 
problems including program communication, staffing, planning, and developing coordinated programs 
in concert with social service agencies. Finally, tl - . ' poses recommendations as they apply to one 
or more levels within the educational system: the Ci-i^om/school, local, and state levels. 



SUMMARY 

■ K-3 PROGRAMS 

I Given the characteristics of K-3 at-risk children, descriptions of program strategies designed to 

meet their needs, and the summary of evaluation activities and results, a picture of at-risk children and 
programs emerges as discussed below. 

• The K-3 at-risk pilot programs, as a whole, have had a positive impact on the lives 
I of at-risk students and parents. 

Arizona's K-3 at-risk pilot programs have implemented a number of educational alternatives to 

■ help children at risk of academic failure. Since their implementation, there has been a steady decrease 

■ in the number of children retained within pilot sites. Additionally, attendance has been on die rise and 
net ITBS gains have been witnessed in language and reading-with third grade students exiting at higher 

I skill levels than their predecessors. Consistently, staff and parents have praised their school's efforts to 

give more individual attention to students and parents alike. Staff have increased their awareness of 
appropriate instructional strategies for working with at-risk (and all) youngsters. Schools have received 

I much-needed funding to add staff and purchase supplies and materials to enrich their students' educational 

experiences. Greater numbers of parents have begun to play a role in the educational system-some 

_ coming to school for the first time since their own adolescence. And, there has been greater accountability 

| among these sites than ever before. 

■ Schools have done much to improve the educational environments of at-risk children; 
I unfortunately, they cannot make "at-riskness" disappear because children are at-risk for reasons that are 

largely environmental and outside the scope of school. There are still barriers to overcome and progress 
I to be made. For example, children need to exhibit greater skills in all areas, particularly math, to advance 

at grade level. Schools need to address philosophical differences regarding retention and "social 
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promotion." Parents of at-risk youngsters need to become involved to a greater extent in the educational 
process. Schools need to establish more collaborative partnerships with social service agencies, and make 
greater efforts to reach out to parents and community members. Staff must continually upgrade their skills 
and keep abreast of the latest research and technology that will enhance their abilities to increase 
individualized instruction. 

At-risk children have shown progress within tht context of the Arizona At-Risk Pilot Project. 
Additional progress is dependent on continued support at the state and local levels. 

• Consistent descriptive and statistical differences suggest that at-risk children, 
programs, and staff v iry regionally. 

Arizona's reservation at-risk children are affected by more at-risk factors than are children in 
other regions. ITBS scores for a select group of these children suggest that reservation at-risk pupils score 
lower on these tests than do their rural and urban peers; likewise, absentee rates are consistently higher. 
Key program staff turnover is higher than in other programs, and staff working in reservation districts 
consistently rate the effectiveness of services (for students, parents, and staff) and outcomes less positively 
than do their colleagues. 

The isolation of these districts compounds their problems. Parent involvement is difficult, given 
that many parents do not have telephones or access to transportation. Unemployment may also be a factor 
contributing to a lack of parent involvement. Staff qualifications pose particular problems, both in 
recruiting qualified staff and keeping them once trained. Providing appropriate training is also problematic 
since it is difficult and costly to recruit qualified trainers to provide on-site in-service; it is equally 
demanding and costly to send staff to other locations for training. Social service linkages are hard to 
establish because of the tremendous demands o.. those services and the tribal infrastructure governing 
most of these services. 

Nevertheless, reservation programs are piloting intervention strategies that hold promise, and at- 
risk children are progressing developmentally. "Language-rich" classrooms and programs emphasize 
literacy skills; full-day kindergartens and smaller student-adult ratios help to provide extra and more 
individualized attention to meet childrens' academic needs. One district sends tape-recorded books home 
with children, who sometimes listen to the stories on their long bus rides home. An additional benefit of 
this practice is that parents are encouraged to listen to (and read) these books as well. Another district 
utilizes cross-grade tutoring integrating instructional and recreational activities. In many districts, parents 
are enticed to school through social events, and more efforts are being made to conduct home visits. 
Finally, through additional training, staif are increasingly aware of the educational and cultural needs of 
their student populations. 

Rural at-risk children in Arizona live between two worlds. They have the isolation of small 
communities., yet access and exposure to metropolitan areas. In virtually every data set, rural at-risk 
children fall in the middle of the extremes represented by reservation and urban children. Rural children 
appear at-risk primarily due to poverty and the conditions that accompany being poor~a lack of 
educational/reading materials, substandard living conditions, parents who work and therefore are not 
home after school contributing to the latch-key status of many rural children. As with reservation 
districts, rural isolation appears to exacerbate low parent involvement and difficulties recruiting and 
keeping qualified staff and administrators. 
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Unlike reservation areas, however, rural communities are increasingly becoming the "crossroads" 
linking metropolitan areas. Therefore, rural areas are being increasingly exposed to problems more 
symptomatic of urban areas. For example, one rural community was described as the "drug connection" 
between Tucson and Phoenix, and one at-risk third grade student was "busted" for pushing drugs on 
behalf ofjiis gang member parents. 

Rural staff perceive moderate success in their programs, but they—like their colleagues-are 
beginning to make a difference. And, more than their urban and reservation colleagues, they are 
optimistic that their efforts to reach parents are paying off. Classes for parents (e.g., ESL and G.E.D.) 
appear particularly promising in these areas, in addition to school social events and better communication. 
Moreover, social services are perceived to be stronger than in the other two regions. Notably, more rural 
districts have piloted "Family Resource Centers" which, in addition to serving children through after- 
school activities and tutoring, also offer parents such services as crisis counseling, prenatal care classes, 
and free legal advice. One center even obtains day-old bread for free distribution. Other schools have 
implemented "clothing exchanges" and parent support groups. These types of programs reflect the fact 
that many of these districts are attempting to address the poverty issues that contribute to the at-risk status 
of their student populations. 

Urban/suburban programs portray yet another picture. Staff consistently express more positive 
attitudes about their efforts and accomplishments than do their colleagues in rural and reservation areas. 
Students function higher than children in other regions, and show more academic progress-at least for 
those children who remain in the system long enough to track. Urban staff are more stable than staff in 
other regions, and it is easier to offer them training given the proximity to universities and community 
colleges. 

Nevertheless, the urban portrait of at-risk children is disturbing for other reasons. Although the 
reported incidence of at-risk factors is not as high for urban children, a profile of poverty and neglect 
still emerges. The effects of parental substance abuse and abusive home environments are particularly 
pronounced for urban children, and this finding suggests that many urban children are most at-risk when 
their home environments are "dysfunctional. " And, more than in other areas, these children are less likely 
to remain in the same school. 

Many urban programs are tackling these problems head-on. One district offers classroom settings 
specifically geared to "welcome" transient students and ease thei. transition to permanent status; other 
districts are actively reaching out to parents through parent liaisons and social workers. Staff are confident 
that full-day kindergartens, reduced student-adult ratios, summer schools, and curriculum modifications 
are making a difference in the lives of these children. 

Overall, these findings strongly suggest the need for local autonomy in program planning, 
implementation, and evaluation so that the unique aspects of at-risk student populations may be 
appropriately addressed. 

• Program implementation issues are key factors affecting the likelihood of program success. 

Much research has shown a relation between "effective schools" and implementation patterns. One 
example is a 1988 evaluation of a California school reform initiative wherein "high gain" schools were 
noted to have "implementation patterns" associated with eight factors: "(1) clear and consistent district 
reform visions...; (2) more active use of cross-role teams and implementation plans; (3) stronger 
implementation coordination between the school and the district...; (4) greater use of initial training; (5) 
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active administrator pressure and monitoring; (6) substantially more on-going assistance, both from 
district and school leaders; (7) stronger on-going administrative commitment and leadership; and (8) tight 
coupling between schools and their districts" (Odden and Marsh in Hannaway and Crowson, p. 55). 
Preliminary comparisons 21 of site evaluations, staff perceptions, and outcome data corroborate this 
research. It appears that successful sites do, in fact, have a clearer vision of their at-risk programs, better 
communication and involvement at all levels, high expectations, teacher commitment, and strong and 
stable program leadership. 

Specifically, effective communication at every level of the organization is essential for successful 
program implementation: when communication is lacking or inadequate, programs suffer. 
Com lunicaticn, including the active participation of everyone who will be affected by an organizational 
change, needs to be initiated early in the change process so that changes are not initiated in a top-down 
fashion. Teachers want to be involved in program planning and decision-making activities, not just in 
implementing programs, and they want time to participate in these activities. New programs must be 
aligned with a well-articulated and shared district mission that has been locally defined through a 
participatory process. 

In addition, schools and socifJ service agencies must collaborate if programs are to effectively 
serve at-risk students. Becaus ,of the extreme nature of the social and emotional needs of at-risk students, 
providing social services has become a de facto function of the schools. Because urban, rural, and 
reservation programs exist in such different contexts and environments, local collaboration is believed 
to be the most effective means of planning how to delive: service/ Social services are most desirable 
when they are based at the school site and provided by qualified social service staff; however, teachers 
need training on how to work with social service providers. By moving services closer to the students, 
social services will also be more readily accessible to their families. 

Strong and stable leadership and administrative support are necessary for programs to succeed: 
lack of this support has been an impediment for many programs. One problem with school leadership has 
been extremely high administrative turnover. When new administrators are placed in programs every 
year, they bring their own personal goals, beliefs, and directions. Often they have not been provided with 
historical knowledge of the programs for which they are responsible, and usually they had no involvement 
in program planning. In some instances, the only program continuity has been provided by a stable 
teaching staff, who tend to become cyr.isal watching administrators come ari go. Turnover of program 
leaders has been a concern for nearly all programs, but has been especially severe in reservation 
programs. Ai incentive structure to create stability at the administrative level should be thoroughly 
explored. 

Several districts that are "ahead of the game" began to address at-risk issues prior to H.B. 2217 
(1988). When these districts received additional funding, they had a clear idea of what kinds of programs 
and activities were needed to supplement their district efforts. At-risk pro£-ams were designed to meet 
specific needs for particular children, and greater efforts were made to integrate these services within a 
"total delivery system." In contrast, Morrison Institute evaluators reported some districts' struggles to 
implement "top-down" programs with ill-conceived objectives, lacking any integration with overall district 
or school initiatives. As a c se in point, one district showing the least gains in ITBS scores is one that 



Individual dutrict data have been examined to determine pattern* of "what work*;" however, a more definitive analyiU of 
individual diitricti' gucceasful programs will be highlighted in an additional report: Promising Practices for At-Risk Youth 
(forthcoming in June i992). 
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has experienced considerable political conflict between the school board, district/school administration, 
program administration, and teaching and support staff. 

These findings suggest the need to create a 'school climate'' conducive to program success. An 
argument can be made in support of preliminary program planning, with technical assistance if needed, 
to involve staff and administrators in developing a comprehensive plan incorporating appropriate training, 
school-cotnmunity partnerships, and on-going monitoring and evaluation. 

• Evidence suggests that the length of time programs have had to implement services 
makes a oiuVence in the evaluation results obtained -the longer the program's 
chance to evolve, the more satisfactory the program implementation. 

This finding refers to observed differences between phase I programs (initiated in FY 1988-89) 
and phase II programs (initiated in FY 1989-90). In virtually all instances, evaluation results were more 
highly positive for phase I than phase II programs, The only significant variable that distinguishes 
between these programs is the length of time they have operated under the auspices of H.B. 2217 (1983). 

it appears that programs need time to evolve. 

• Key early childhood student service strategies identified as "promising" include: 
reducing staff-student ratios, implementing full-day kindergartens, and 
supplementing individualized instruction. 

Reduced staff-student ratios, full-day kindergartens, and tutorial programs (particularly those 
delivered during the school day) consistently appear as "effective" strategies holding promise for at-risk 
youth. National research also supports this finding. These strategies share one critical feature: children 
receive more, and more individualized, time and attention. 

Although the interpretation of class size research has been highly controversial, a recent meta- 
analysis of cl? s size research reveals that smaller classes result in higher student achievement, and these 
effects are cumulative (Mitchell <k Beach, 1990). rurther, large scale experimental studies in Tennessee 
show that the positive effect of small class size for minority students in primary grades is twice that for 
whites (Finn & Achilles, 1990;. The minority factor is particularly germane to the at-risk study because 
over 80 percent of the students served through Arizona's at-risk programs are ethnic minorities. Because 
reducing class size on a large scale has a significant price tag, educators are now searching for ways to 
reduce class size using available resources. Mitchell and Beach describe strategies such as "redeploying 
staff" for designated parts of the school day or "redistributing students" through creative grouping for 
some instruction. The use of classroom aides can also impact class size ratios. 

Extending the traditional half-day kindergarten schedule to a full day appears to be another 
promising practice for serving at-risk students. A recent review of the research on full-day kindergarten 
(Puleo, 1988) revealed that, although many research questions remain to be investigated on this practice, 
th available evidence largely favors full-day programs over half-day, particularly for lower ability and 
k o-economic status (SES) students. Additionally, achievement effects are shown to be long-term, 
u la. arable findings continuing when students are followed into the upper grades, even as high as 
e. b i -ade. 

Teachers also believe that tutorial programs are effective, but only if implemented under the 
appropriate conditions. For example, before- and after-school tutorials have not been as well received as 
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during-school programs. If they are conducted during-school, teachers prefer that tutoring take place 
within the classroom setting rather than on a pull-out basis. Several other aspects of delivering tutorials 
must be considered if an effective program is to be implemented: the frequency and intensity of the 
tutoring, the process by which students are identified, and the skills identified for tutoring. 

In general, "promising practices" for at-risk children appear to be those that increase both 
individual attention and instructional time for students. Notably, these are also costly reforms. 

• Developmental))' appropriate practices (DAP) appear promising, but evidence is 
inconclusive as yet. 

The use of DAP is desirable and well supported in the early childhood literature and research 
(National Association for the Education of Young Children, 1985; Bredekamp, 1987). As defined by the 
NAEYC, these types of practices address the total educational environment of children including 
curriculum teaching strategies, social-emotional development, motivation, parent-teacher relations, the 
physical environment of classrooms, evaluation techniques, and class size, among other factors. 

Within Arizona, virtually all K-3 at-risk pilot programs focused on the increased use of DAP 
through implementing curriculum modifications incorporating "whole language" and experiential math 
programs. Many programs used at-risk funds to purchase the instructional resources needed to implement 
DAP, such as classroom libraries, books with audiotapes, and math manipulative materials. Teachers are 
extremely positive about the benefits of these types of enriching materials for the students. 

However, the actual effectiveness of DAP in Arizona's at-risk pilot programs was difficult to 
assess because of wide variations in the extent to which they were understood, employed, and 
implemented. In addition, DAP tended to be incorporated into other intervention strategies making it 
difficult to separate them for analysis. As a result, Morrison evaluators, ADE personnel, and some 
district personnel have expressed concerns regarding apparent inconsistencies in the understanding and 
implementation of DAPs. Further district efforts are required to enhance uniformity of practice so there 
can be meaningful analysis of whether or not these practices "work" with Arizona's at-risk pilot students. 

The implementation of developmental^ appropriate practices needs to be assessed further. 

* Research findings suggest two distinctly different aspects of "effective" parent 
involvement: 1) parent support, and 2) parent training. 

Efforts to involve parents were required as part of the K-3 at-risk legislation, and all programs 
did indeed offer parent involvement components. Overall, parental involvement is believed to have 
improved since the initiation of the at-risk programs, but much more progress is needed and desired. 

From the at-risk evaluation, much was learned about the lives of at-risk children and their 
parents. For example, many parents exist in a cycle of poverty and lack the most basic necessities. By 
and large, they are not educated, and speal: English poorly if at all. Most care about the ; r children, but 
do not feel comfortable in the school environment because they themselves met with failure there. 

One key finding centers on teachers' perceptions of parental support. Although lack of parent 
participation is considered pervasive, it does not have as negative an impact on a child's achievement as 
lack of support. Therefore, garnering parental support becomes a de facto role of schools attempting to 
improve the educational outlook for their at-risk students. Evaluation results indicate that the most 
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effective means of getting parent participation is through social events where food is served. But, to build 
parent support, verbal, one-on-one communication from the classroom teacher or another staff member 
is the best strategy. 

Parent workshops, particularly of the "hands-on" variety (e.g., make-and-take), have been shown 
to be somewhat effective. These workshops, however, tend to reach a relatively small number of parents- 
often not the ones who need the contact most. In rural areas, classes that upgrade parents' own skills 
(e.g., ESL and G.E.D.) have met with some success. 

In sum: 1) Parent involvement initiatives, targeted specifically toward parents of at-risk youth, 
should first consider the existing level of parent support. Garnering support seems a necessary 
prerequisite for parent involvement. Activities that promote school-parent rapport and establish schools 
as comfortable, non-threatening environments may initially hold more promise than parent workshops or 
other parent training activities. 

2) Workshops that require the active participation of parents are more successful than those that 
merely present information; adult education classes also hold promise for eliciting parent support and 
participation. 

• At-risk pilot sites must have the "right" people for programs to succeed-those with 
appropriate training and commitment to work with at-risk youth. 

Survey data, open-ended responses, and interview data on staff services make a strong case for 
the conclusion that quality and commitment of staff are key aspects of program success. Yet data also 
suggest that staff pre-service and in-service training often fail to prepare staff for the challenge of working 
with at-risk populations. Program personnel recommend that pre-service training include more and earlier 
practical teaching experiences in diverse settings, including those with at-risk populations, and that 
students be encouraged to exit undergraduate programs with ESL certification. Internships, mentoring 
programs with master teachers, and five-year undergraduate programs are additional options for 
improving pre-service programs. Other district recommendations include more collaboration and closer 
linkages between schools and universities, and greater input establishing teacher training requirements. 

Quality on-site in-service training is often difficult and costly to provide. Making better use of 
available communication technologies could alleviate some of these problems especially in rural and 
reservation area. Suggested incentives for promoting professional development include: giving teachers 
control of resources at the building ievel, paying for college courses with the stipulation that teachers stay 
with the district for a designated number of years, providing stipends for teachers who fulfill a staff 
development function in the school, and paying teachers to attend training offered during the summer. 

Staff training-both pre-service and in-service-is an area in need of state and local attention. In 
particular, strategies for providing training in more isolated districts need to be explored further. 
Additionally, strategies to reduce turnover should be developed in all programs. 



7-12 PROGRAMS 

Given the characteristics of 7-12 at-risk pupils, descriptions of program strategies designed to 
meet their needs, and a summary of evaluation activities and results, a porirait of at-risk youth, staff, and 
programs has unfolded. The major findings of our analysis of 7-12 at-risk pilot piograms are: 



119 




1 or 

A. vi. 



• The 7-12 at-risk pilot programs, as a whole, have had a positive impact on the 
lives of dropouts and potential dropouts. 



Arizona's 7-12 at-risk pilot programs have implemented a number of educational alternatives to 
help dropouts and potential dropouts. During FY 1990-91 alone, nine out of every ten program 
participants reportedly remained in school or had graduated. Attendance rose for the first time since the 
inception of the programs. ITBS gains have been made in language and math-with twelfth grade students 
exiting at higher skill levels than their predecessors in these areas. Students and parents alike have 
attributed a number of attitudinal and behavioral changes to program participation, often crediting helpful 
and caring staff with providing more individual attention. Staff and students, particularly at the junior 
high school grades, note increased parent involvement and participation in the programs. Much-neeawd 
funding has allowed alternative programs and activities to develop and expand, and there has been an 
unprecedented level of accountability associated with this funding. 

The 7-12 at-risk programs have continued to meet the challenge of working with teens who are 
well acquainted with school failure. Many of these students have entered these programs lacking even the 
most basic skills-therefore, progress reflected in this evaluation must be looked at relative to where these 
students began. For many district staff, however, this progress is simply not good enough, resulting in 
mixed reviews of program effectiveness. Unlike K-3 programs in which there is a prevalent compassion 
for the plight of at-risk children, 7-12 programs frequently are embroiled in controversy-caught between 
the extremes of staff opinions regarding appropriate educational and support services for at-risk youth. 

There is no doubt that at-risk teens need to exhibit greater skills in all areas, and reading in 
particular. But more important perhaps, schools need to address philosophical issues as: the comparability 
of curriculum between "traditional" and "alternative" courses of study, desirable types of delivery systems 
for at-risk youth, and whether or not to actively solicit parent involvement. As do their K-3 counterparts, 
7-12 schools need to establish «iore collaborative partnerships with social service agencies, and make 
greater efforts to reach out to the business community. Staff training needs to target non-program staff 
as well program staff. Also, more staff are needed-not only instructors, but counselors and soctel 
workers as well. 



At-risk youth have shown progress within the context of the Arizona At-Risk Pilot Project. 
However, there are a variety of program improvements which must be made to further enhance the 
effectiven of these programs. 

• Consistent descriptive and statistical differences suggest that 7-12 at-risk youth, 
programs, and staff vary regionally-and in patterns somewhat different from their 
K-3 counterparts. 

In the K-3 at-risk pilot programs, reservation children were found to be "most" at-risk based on 
the prevalence of at-risk indicators. In contrast, for the 7-12 at-risk pilot project, urban program youth 
are characterized by the most indicators. Urban 7-12 at-risk youth represent a broader spectrum of ethnic 
representation than youth in other regions and tend to be from the least "nuclear family-oriented" 
households. More urban youth have children, have a sibling who has dropped out of school, have 
dropped out themselves, indicate poor health, have seriously considered or attempted suicide, skip school, 
been suspended or expelled, been convicted of a crime, and have parents who are neither supportive of 
nor involved in their education. 
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Key personnel among urban programs are more stable than in rural or reservation areas. Urban 
staff, however, are more negative regarding program implementation and more reserved in their 
assessment of program outcomes. While 80 percent believe they are contributing to at-risk teens' progress 
toward completing high school, 60 percent feel they are watering down the curricula in order to do so. 
Nevertheless, although urban teachers appear somewhat pessimistic about their accomplishments, urban 
students do not appear to share this perception. These students, more than their rural and reservation 
peers, view program staff as helpful and caring, and credit their programs with involving their parents 
more with their education, improving their grades, helping them stay in school, helping them set future 
goals, and increasing their self-esteem. With respect to the latter, it is noteworthy that more urban 
programs have incorporated specifically-designated social/support activities as part of their at-risk 
programs than have other regions. 

External evaluators, in consulting with program personnel, hypothesize that one reason for 
negative urban staff perceptions of their programs may be that these programs have tended to implement 
discrete activities that are more difficult to coordinate and integrate within a total delivery system. In 
addition, because pupils are "mainstreamed" for most of their schooling, they are subject to other non- 
program staff members who may not have the training and/or compassion to "deal" appropriately with 
this population. Although a majority of the discrete components are fulfilling their specific purpose (e.g., 
vocational components result in positive vocational outcomes), it may be difficult to produce wide-ranging 
outcomes from "one-shot" intervention strategies. 

Rural at-risk youth in Arizona's pilot programs represent a more mobile population and a greater 
percentage of recent immigrants to the United States. It is not surprising, given this, that more at-risk 
rural youth speak a language other than English at home and have language difficulties in school. They 
tend not to be involved in any school/community activities, and have a history of academic failure (with 
one or more grade retentions in their elementary years). Relatively more rural youth use drugs and/or 
alcohol on a weekly basis. 

Conspicuously, rural staff and students are the most positive of all regions regarding virtually 
every aspect examined at the 7-12 level. Staff are positive regarding program implementation, delivery 
systems, all student services, and program outcomes. And, students in these programs which are largely 
"holistic" in their approach to at-risk pupils, are more likely to attribute changes in behaviors and 
attitudes to their participation in them. Moreover, rural program students are out-performing their peers 
academically. 

Finally, reservation youth appear to reflect "at-riskness" more on the basis of life circumstances 
over which they have little control than because of behavioral problems. As profiled, more reservation 
students indicate that they have responsibilities that interfere with school work a*.J live in homes that do 
not have year-round electricity, plumbing, and/or telephones. Most disturbing, more reservation pupils 
do not feel safe and/or protected at home. In fact, a majority of program directors have estimated that 
three out of every four reservation pupils is from an abusive home environment. 

As previously mentioned for their K-3 counterparts, the isolation of reservation districts 
compounds their problems. Parent involvement is difficult, given that many parents do not have 
telephones or access to transportation. Staff qualifications pose particular problems-both in recruiting 
qualified staff and keeping them once trained. Providing appropriate training is also problematic-it is 
difficult to recruit qualified trainers to provide on-site in-service; it is equally demanding and costly to 
send staff to other locations for training. Social service linkages are hard to establish because of the 
tremendous demands that currently exist on the services an J because of the tribal infrastructure governing 
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most of these services. It is perhaps not surprising, then, that reservation staff are least positive regarding 
many aspects of their programs. 

Regional differences, although distinct from those found among K-3 programs, suggest a similar 
conclusion-that there is a need for local autonomy in program planning, implementation, and evaluation 
so that the unique aspects of at-risk student populations may be appropriately addressed. 

• Consistent descriptive and statistical differences suggest that at-risk yeuth in 
grades 7-8 differ from their 9-12 peers. 

A majority of the seventh and eighth grade students examined within this evaluation study are in 
rural and reservation areas, and for this reason, the demographic profile obtained may be more suggestive 
of regional differences than age/grade differences. Nonetheless, students in grades 7-8 are almost 
exclusively minority (93 percent) in contrast with their high school counterparts (75 percent). A greater 
percentage indicate that they live with both natural parents and speak a language other than English at 
home (60 percent versus 40 percent at the high school level). There are proportionately more females 
included in this population, and more students whose mothers are unemployed outside the home. A 
slightly greater percentage of junior high than high school students indicate that they do not feel safe 
and/or protected at home. Overall, however, students in the earlier grades exhibit fewer indicators of "at- 
riskness M than their 9-12 peers. 

The image that emerges from a collation of all data is that these students are truly at a stage in 
their development where they could "go either way." It may be of practical significance that there are 
more indicators potentially predictive of academic success among these students than for high school 
students-notably, a majority of these indicators suggest emotional-behavioral problems (not involved, 
drug/alcohol use, suspension, juvenile delinquency). This implies that interventions have a better chance 
of "preventing" school failure. 

The interventions that have been studied primarily involve supplemental services, although there 
are several school-within-school models. Notably, junior high students responded most positively to social 
support: these kids need people to care about them. Moreover, junior high survey respondents are in 
greater agreement that program participation has prompted their parents to help them more with 
schoolwork, and that programs have provided more opportunities for parents to become involved. Staff 
for grades 7-8 reflect this same attitude, expressing greater satisfaction with parent involvement than their 
9-12 colleagues. In fact, junior high staff are more positive than their colleagues regarding program 
implementation, outcomes, and staff training. 

On the other hand, junior high students are more likely to indicate that staff do not care and/or 
are not helpful. Interview and anecdotal data suggest that some 7-8 staff are more likely to adopt a 
"tough" disciplinary stance with these students. At the same time, these students are at a stage where they 
are-perhaps more than ever-questioning authority and responding to peer pressure while at the same time 
trying to maintain strong ties with adult role models (Task Force on Education of You Adolescents, 
1989). Some reconciliation between positive staff views on their involvement and negati student views 
seems desirable. 

Evaluation findings have not provided definitive answers regarding successful interventions for 
junior high students; however, there are some components perceived as "making a difference" in the eyes 
of students. Urban students respond to support groups and a self-contained class that provides academic 
and social support as well as opportunities for work experience. Rural pupils respond to school-within- 
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school settings, which allow for more individualized attention and self-paced curriculum. And, reservation 
youth respond to vocational activities that incorporate applied academics and social support. 

More intervention programs are desirable at grades 7-8 because at-risk behaviors are not well- 
defined as yet. Programs may, therefore, have more impact in "curbing" the further development of at- 
risk behaviors among early adolescents. However, such programs should be implemented by qualified staff 
who are committed to working with this challenging population. 

• Program implementation issues, similar to those identified for K-3 programs, are 
key factors affecting the likelihood of program success. One issue in particular is 
more pronounced at the 7-12 level, however, and needs to be more systematically 
addressed: philosophical differences among 7-12 staff regarding at-risk students and 
programs. 

All program implementation issues discussed earlier in this chapter with respect to K-3 programs 
hold true at the 7-12 level as well. However, philosophical differences among staff in their attitudes 
toward at-risk teens and programs are much more pronounced at the 7-12 level. These differences are 
evidenced by numerous descriptive and statistical differences among 7-12 staff in all data sets, and in staff 
survey responses in particular. 

Differences between administrators and staff were quite apparent. Administrators consistently gave 
more strongly positive responses than did their teaching/specialist staff. This may reflect the optimism 
and support of administrators and, in fact, select data suggest that such support and program leadership 
are considered major contributors to program success. Nonetheless, not all administrators have been 
perceived as being "in touch" with the grass roots. Given the nature of administrative turnover, this 
appears to be a valid perception in some cases. The disparity between administrative and staff viewpoints 
is of some concern. 

Differences between program and non-program staff were also significant and are more troubling. 
It was anticipated that program staff would be more positive than non-program staff regarding program 
efforts; however, it was not anticipated that one of every ten staff members surveyed had no knowledge 
of their district's programs for at-riskyouth. In addition, many non-program staff expressed opinions that 
were unsympathetic regarding at-risk youth and against at-risk programs (e.g., "At-risk kids are simply 
trouble-makers looking for easy ways out;" "Our [programs] and policies are a joke;" "It should be 
called: How to earn a credit in 3 easy minutes!"). 

Data show more consensus among staff at sites which implemented self-contained programs (e.g. , 
off-site alternative schools). In addition, a greater percentage of students i i these programs say they feel 
"respected" as human beings, despite their reputations. Notably, more "ho istic" changes in attitudes and 
behaviors have been observed among students in these delivery systems. Many at-risk teens say they feel 
more comfortable in these setting* because they are not as "different." 

In contrast, more staff discrepancies are documented in districts implementing more discrete 
interventions (e.g., one class a day). Program staff may feel confident that they are making progress with 
their pupils; among all staff, however, there is more diversity in attitudes and little consensus regarding 
"what works" for at-risk teens. It is obvious that within such delivery systems, at-risk pupils interface 
with non-program staff members for a significant proportion of the day. Perhaps it is because some staff 
members have negative opinions of at-risk students that students in discrete interventions express fewer 
changes in attitudes and behaviors. 
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Research has suggested that successful dropout programs create a school climate that is secure, 
safe, and comfortable. As Hamby states: "The emotional atmosphere must be positive so that students 
will not fear a loss of self-esteem by being there" (1989, p. 83). To the extent that philosophical 
differences among 7-12 staff reflect theirs behavior toward at-risk students and programs, students 
undoubtedly receive "mixed messages." Evaluation of Arizona's at-risk sites demonstrates that a cohesive 
philosophy among staff in the more holistic programs is associated with more positive attitudes toward 
program outcomes among students. Programs in which staff have notable philosophical differences 
associated with more negative student perceptions. 

Findings parallel K-3 evaluation results in many respects, and suggest the need to create a 
"school climate " conducive to program success. At the 7-12 level in particular, greater efforts should be 
implemented to achieve consensus regarding at- isk youth end appropriate interventions. 

• In the context of this evaluation, alternative programs appear most "promising" 
for at-risk 7-12 students; however, each type of student service strategy may be 
described as "promising" in relation to its specific intent (e.g., to provide vocational 
services). 

This report has repeatedly noted the diversity of programs implemented at the 7-12 level. This 
diversity has added to the complexity of the evaluation efforts undertaken and has produced findings that 
suggest that each type of program has some specific merits. For example, alternative and school-within- 
school students have rated behavioral and attitudinal outcomes positively "across the board;" academic 
components produce positice academic outcomes; vocational components produce positive vocational 
outcomes; and support service components produce positive outcomes related to self-esteem and coping 
skills. Which practices are most promising? It depends. 

The Holistic Models 

1) Alternative Schools: Evaluation results consistently point to the alternative school model as 
the most effective and positively perceived delivery system for at-risk students at the secondary school 
level. By incorporating a variety of instructional, vocational, and support strategies into a comprehensive 
system, customized to the unique and diverse characteristics of older at-risk students, alternative schools 
seem to be providing a viable option for students who have been disenfranchised from *Jhe "regular" 
educational system. While mainstreaming and heterogenous grouping are important goals for serving at- 
risk students at the primary level, this approach may not be as desirable for older students who appear 
to function well when grouped with student similar to themselves. 

The "credibility" issue is important in looking at the alternative school, since there is a perception 
among non-alternative school staff that they offer an "easy out" for students who have not succeeded in 
the mainstream. High academic standards and quality curriculum must be in place if alternative school 
are to be viewed as a positive intervention for at-risk students. Regional differences are also significant, 
since seven of the nine alternative schools in the pilot programs are in rural areas. However, alternative 
schools have been quite well established in urban areas for many years, and it may be happenstance that 
only one urban pilot site employs the alternative school model. Further, alternative schools may not be 
the best delivery system for reservation districts where the majority of the population is considered to be 
at risk. But the concept of an integrated, versus a fragmented, delivery system such as that provided 
through alternative schools deserves consideration. 
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I 2) Schools-within-schools (SWS): The SWS model provides educational programming that is a 

compromise between alternative schools and mainstreaming programs, and Iheir effectiveness is also 
_ shown to be somewhere in the middle in terms of student and teacher perceptions as well as student 

| achievement. Students, however, rate behavioral and attitudinal outcomes consistently positive, and are 

the only group of students to do so other than alternative school students. 

I 3) School-wide reform: Another holistic approach being implemented in two at-risk programs 

is school-wide reform. Only one of these efforts was formally assessed as part of this evaluation, a 
I reservation program that instituted the four-period day during year three of the project. This school 

1 reform model has affected every student in the high school, all of whom are considered to be at risk, as 

well as every staff member. After the first year of implementation, responses to the change have been 
I positive overall, with students somewhat more positive than staff. This totally integrated approach to 

serving at-risk students warrants close examination and may be particularly applicable for reservation sites 

with large at-risk populations. 

The "Discrete Intervention" Models 

I 1) Academic and Instructional Strategies: Academically-focused activities do produce feelings 

of academic success among students. However, there was no consensus between students and teachers 

■ about what constitutes effective instructional strategies. Students prefer self-paced instruction, while 
' teachers believe that this method does not provide adequate direct instruction. Computer-assisted 

instruction appears to be effective when used as one aspect of a comprehensive program, but not as a 
I stand-alone strategy. Tutorials are well-received, but again must be planned as a part of an integrated 

system. In sum, each separate instructional strategy appears to be effective only in terms of how well it 
is incorporated into a total synergistic system. 

2) Vocational Services: The wide range of vocational-oriented services that have been 
implemented has resulted in improved attitudes and abilities in work-related areas. Students enrolled in 

I vocational programs report increased awareness of career opportunities and feel better prepared to enter 

the workplace than students not enrolled in these programs. Although staff believe they are somewhat 
I effective, they also feel that the opportunities are limited and sometimes difficult to access. 

3) Support Services: More is the word. Support services are viewed relatively positively and are 

■ resulting in more positive attitudes among the students receiving them, particularly among students in 
grades seven and eight. More qualified social service staff are needed to serve more students in a timely 
manner. Because seventh and eighth grade at-risk students do not show as severe a level of at-riskness, 

I and respond positively to support services, support services efforts should be concentrated at the middle 

school level. Perhaps more severe at-risk behaviors could be averted by intervening earlier with more 
support. 

Since clear conclusions regarding the most effective strategies for secondary level at-risk students 
I were difficult to establish, Morrison Institute staff revisited national at-risk literature, specifically to 

I review studies pertaining to interventions similar to those evaluated in the Arizona At-Risk Pilot Project 

(e.g., alternative schools, tutorial programs, and vocational and social services). Throughout the national 

■ literature, similar research problems were documented (e.g., limited access to longitudinal quantitative 
' data on student outcomes) and inconclusive results were reported regarding "what works" (Catterall & 

Stem, 1986; Gold & Mann, 1984; Reilly, 1986). Much of the research validated such notions as: 1) 
I implementation processes need to be assessed, and 2) attitudinal changes are important student outcomes. 

The conclusion of the Gold and Mann study is particularly relevant to the present at-risk evaluation: 
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"Whether programs are successful will vary from student to student, teacher to teacher, program 
to program... But exploration of the underlying processes has shown that those programs did 
indeed produce marked change in particular attitudes and perceptions of certain identifiable 
students in a way conducive to better behavior and greater scholastic achievement." 

Evaluation findings strongly suggest the need for more "holistic" approaches to working with at- 
risk teens. While such self-contained programs hold promise, however, they may not be realistic options 
in some areas. Discrete services can be effective, if "impact" is sought in a discrete skill. Nonetheless, 
a better solution for implementing discrete services is greater systemic integration of services for meeting 
the needs of the "whole" student. Again, local autonomy in planning and implementation appears to be 
critical. 

• Parent participation is problematic at the higher grade levels. 

The older the student, the less parent involvement, and the lower the expectations that schools 
should expend resources to garner parent support and involvement. Some staff firmly believe thai schools 
should be reaching out to more parents; others at e adamantly opposed to spending their time and energy 
on what they perceive as "not my job." Still, the same strategies that work in the primary grades work 
at the secondary level; namely, holding school events and making personal verbal contacts. Parents of 
middle school students are more likely to respond to these strategies than are parents of high school 
students, and the middle school students are more likely to realize the benefits of increased parental 
support. 

It should be pointed out that in some respects, efforts-or increased efforts-to involve parents of 
7-12 students resulted from the at-risk program evaluation. Many districts did not emphasize this aspect 
of program development, but this was not a concern because they were not required to do so by law. 
However, to accommodate those districts that did include some parent outreach, uniform reporting 
measures were developed and distributed by Morrison Institute to all sites. Several sites expressed the 
notion that if they had known that they were to be evaluated on parent involvement, they would have 
done something more-and, in fact, they did. 

Parent involvement is weak among 7-12 programs. Given that parent participation is socially 
perceived as desirable (but that there is often faculty opposition to investing energy in such activities at 
the upper grade levels), it may be worth developing initiatives specifically targeted for secondary level 
parent activities. Based on K-3 program evaluation findings, it seems prudent to point out that any 
initiatives should first take into consideration the level of parental support for public instruction since 
garnering support appears to be a necessary prerequisite for parent involvement. 



• As with K-3 programs, 7-12 at-risk pilot sites must have the "right" people-those 
with appropriate training and commitment. 

Results regarding staff training parallel those found at the K-3 program level (see earlier 
discussion) and as such, our conclusion is the same: 

Staff training-both pre-service and in-service~is an area in need of state and local attention. In 
particular, strategies for providing training in more isolated districts need to be explored further. 
Additionally, developing strategies to reduce turnover should be explored among all programs. 
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PILOT PROJECT AND EVALUATION PROCESSES 



The demand for greater accountability coupled with limited resources has resulted in a recent 
I trend to create "pilot" projects by Arizona policy makers. The idea is sound: distribute funds based upon 

competitive responses to a request for proposal (RFP), require an external evaluation, and ultimately 

■ make decisions regarding permanent funding based upon project outcomes. Indeed, this is the process 
I adopted for the Arizona At-Risk Pilot Project. 

■ The scope of this project was extensive since it included the monitoring and evaluation of 

■ programs within over one-quarter of Arizona's most "at-risk" districts. Much has been learned by the 
evaluators, by the pilot sites, and by the Arizona Department of Education, regarding the arduous task 

I of monitoring and evaluating long-term, large-scde piograms. In retrospect, many "systemic components" 

necessary to complete the project in the most efficient and effective manner did not exist at the time the 
pilot was initiated-streamlined state RFP procedures, a Department of Education infrastructure for 

I program monitoring, valid performance-based student outcomes measures, and a funding cycle that 

encouraged and allowed long-term planning. 

| This section reflects upon insights resulting ftom an analysis of the overall pilot and evaluation 

process itself that have implications for future at-risk programming and funding. 

I • Adequate program planning was not evident in a majority of district proposals; 

however, adequate time for planning was not available prior to receiving program 

■ funds. 

Initial H.6. 2217 (1988) funds were appropriated in June 1988 for use during the school year 
I beginning August 1988. However, prior to the distribution of funds it was necessary for the Arizona 

Department of Education (ADE) to develop a RFP process and to determine the relative "at-riskness" of 

each district in the state. Once these procedures were completed, districts had about six weeks to develop 
I their proposals. Although some technical assistance was provided by ADE, a lack of program 

planning/grant writing expertise coupled with the short time frame resulted in very little "creative" 
- planning. Further, while the RFP required the development of specific program objectives, most 
I proposals included objectives that were nebulous, unrealistic, and unmeasurable. Primarily because of 

time constraints, proposals were accepted despite their poor quality, and funding was distributed. 

• Generally, timing and planning concerns continued throughout the project. Program reapprovals 

could not occur until annual legislative appropriations were secured (usually late June). The reapprovals 

I then needed to be reviewed by ADE personnel and placed on the State Board agenda for final approval. 
Short turnaround times resulted in many districts simply resubmitting their original proposals with only 
date changes, thus perpetuating a lack of long-range planning and program refinements. 

A lack of adequate long-term planning has had several consequences. Some programs have 
m developed in a "piecemeal" fashion, lacking integration within the district/school. Other programs have 
I failed to involve in planning the personnel who are charged with implementing the programs. Such "top- 
down" initiatives have often been difficult to implement effectively because they lack the support of the 

■ entire faculty. Initial limitations in the proposals (e.g., regarding objectives) were never adequately 

■ addressed in some cases, resulting in a lack of local program focus and systematic evaluation. 

■ Why h planning important? Evaluation data reveal that sites that had developed more 
comprehensive plans based upon extensive input from staff did demonstrate greater outcomes. District 
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and school personnel have acknowledged repeatedly the importance of initial planning and on-going 
review and adjustments. Those who conducted self-evaluations were better aware of program adjustments 
that needed to be made, and produced better results over time. 

District and school personnel have also acknowledged that although planning is important, it 
frequently does not occur without some outside requirements or "pressure." Further, many personnel do 
not like to invest time in planning without some guarantee that their time will be worth the effort (i.e., 
the grant money will be available). Finally, many districts simply need technical assistance in long-term 
planning. Although the movement in Arizona and across the nation is away from state "requirements," 
even the Governor's Task Force for Educational Reform has recommended that schools develop plans 
prior to the receipt of at-risk funds. 

Districts and/or schools should be required to develop comprehensive program plans. Based upon 
feedback from districts, this planning period should be from three to six months. Technical assistance 
from the state department and/or planning monies is/are essential. 

• The current funding cycle, dependent upon annual appropriations rather than 
formula funding, does not promote long-term planning and effective programming. 

As noted in the previous discussion, late adjournments of recent legislative sessions have resulted 
in the authorization of funds occurring only weeks prior to the beginning of the next school year. This 
time frame has effectively curbed district/school efforts to create long-range plans involving at-risk 
monies. Beyond influencing planning, however, the funding cycle has also affected program 
implementation. 

Once funds were authorized, the flow of paperwork from the district to the Department of 
Education to the County School Superintendent's Office and then back to the district resulted in funding 
delays-up to six and seven months in some cases. Some smaller districts could not afford to count on 
the fact that the "check is in the mail;" funding delays sometimes prevented services from being offered 
and, ultimately, affected the quantity and quality of program outcomes available for study. Although 
greater efficiency in processing paperwork could alleviate some problems, a more fundamental issue 
centers on the annual funding cycle. Annual legislative appropriations, used as a funding mechanism, 
simply are not conducive to long-term planning and efficient implementation. 

The alternative is to create permanent funding for at-risk programs through formula funding. 
However, unlike most existing formula funds that are "block grants" (i.e., unrestricted funds), interviews 
with program staff have repeatedly revealed that "targeted" funding for at-risk students is preferable. 
Unless funding is targeted, districts report, there are too many demands on their budgets and not enough 
advocates for at-risk students when it comes to budget decisions. Pilot districts want the accountability 
associated with targeted funding 29 ; however, they do not want extensive restrictions on the use of these 
funds. 

"Dure is a need to establish permanent funding for at-risk programs, to ensure program 
continuity. Direct formula funding "targeted" toward at-risk programs is a preferred funding mechanism. 



^Unlike, for example, K-3 direct formula fundi that provide direct funding without any state rcitrictiona/requiremenU. A 
wealth of anecdotal evidence it available to ahow that the Si 00 plus dollar* currently being provided for each K-3 itudent would 
have resulted in greater outcomes if targeted toward those students. 
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• The identification of district/schools as "at-risk" and the determination of grant 
amounts were appropriate for a competitive grant process; however, new 
mechanisms need to be created if permanent formula funding is utilized. 

Consideration needs to be given to the criteria used as part of an "at-risk weight." Several weights 
currently provide additional funds for students who are identified as having certain characteristics (e.g., 
handicapping condition, LEP). The definitional characteristics for these weights are very specific in order 
to avoid labeling students for funding purposes. "At-risk," however, is a term not easily defined since 
there are a variety of reasons a student may be at-risk (e.g., poverty, teenage pregnancy, drug abuse, 
broken homes, drug abuse, limited parental support). Unfortunately, reliable and comparable data on most 
of these indicators are currently not available. In addition, it is important not to base funding on 
indicators over which districts/schools have some control (e.g., low test scores, absenteeism). If these 
factors are utilized, then improvements may result in a funding loss. 

As a solution, indicators which cannot be manipulated and which serve as a "proxy" for at- 
riskness are being used in several states. New Jersey and Kentucky are using poverty, as measured by 
eligibility for federal free lunch programs, as their at-risk weight proxy. Poverty is frequently chosen 
because of its underlying linkages to at-riskness 30 . This does not mean that only poor students can be 
served with the at-risk weight funds, but instead it provides a mechanism for providing additional funding 
to the schools. Once the funding is received by the district/school, they are allowed to serve those 
students they believe to be at-risk. 

Within Arizona, preliminary discussion among several state school finance experts has resulted 
in the identification of three indicators that have established databases and represent indicators over which 
districts have little control: poverty, mobility, and LEP. Since the state already partially funds a LEP 
weight, the current thinking is that only two factors would be used to establish an at-risk weight-poverty 
and mobility. 

Two options for the measurement of poverty appear to be available and are being analyzed. One 
option is using census data (similar to federal Chapter 1 funding). However, using census data poses 
several problems and/or concerns: 1) necessary data will not be available until at least 1993, 2) data will 
not be available at the school-level, and 3) the database is updated only once every decade. The second 
option is to use the number of eligible students for the federal free lunch program. This too has problems 
since many high schools and some elementary schools do not offer such a program. In addition, the 
percentage of families that actually apply are fewer than those who are eligible. However, the advantage 
is that these data are available at the school-level and are updated annually. Continued work on the 
mechanics of funding is underway. 

Finally, consideration must be given to an "adequate" level of funding. For the pilot grant 
project, districts/schools were allowed to request any amount within a certain range. If an at-risk weight 
is utilized, however, a specific amount needs to be placed in the formula. This amount should be no less 
than the average expended per student by these pilot programs (e.g., $227 for K-3 and $251 for 7-12). 
In addition, dependent upon the expectations for training, parental involvement and social service 



x The uk of at-risk indicator! as part of formula weighting ia currently being analyzed by Dr. T.S. Lyons from the University 
of Nevada at Lai Vegaa and Dr. K.F. Jordan from Arizona State Univeriity. Their findinga from regreaaion analyse* ahow that 
of eleven at-riak indicator! currently used in Texas, one indicator-poverty-accounted for 90 percent of the variance, while mobility 
accounted for the next 1.5 percent of the variance, and LEP accounted for 0.4 percent (telephone interview with K.F. Jordan, 
October 1991). 
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coordination, the amount should be higher than the average. For example, Levin (1989), an economist 
from Stanford University, estimates at-risk funding should be about half again as much as is spent on 
nondisadvantaged students (e.g., approximately $2,000). In Kentucky, $528 per free lunch student was 
provided during FY 1990-91 for both general programming and for Family Resource Center support. 
Since nearly one-third of Arizona's students qualify for free lunch status, these amounts may not be 
realistic; however, they are offered as a starting point for discussion, 

Lastly, additional funding (over the base weight amount) should be provided to small and rural 
schools, and for those programs that wish to develop more comprehensive services such as alternative 
schools. A minimum amount (e.g., $5 ,000-$ 10,000) should also be considered for all except the very 
smallest of schools. 

An at-risk weight based upon poverty and mobility indicators would provide a mechanism for 
directing funds into a district/school; local decisions would then be made at that point as to which 
students are to be served. Additional funding should also be considered for small and rural schools and 
for more comprehensive services. 

• Arizona Department of Education technical assistance efforts contributed to 
project outcomes; however, the current state-level infrastructure for monitoring 
efforts needs continued refinements. 

The Arizona Department of Education contracted with Morrison Institute to conduct the evaluation 
study, but retaineJ the functions of program monitoring and technical assistance. As a whole, the pilot 
programs responded very positively to the available technical assistance (especially at the K-3 level, 
although ratings for the 7-12 support improved over time). In fact, the greatest concern is that "more" 
was needed, particularly in reference to the initial grant writing process and on-going program 
refinements. Since many of the at-risk programs are located within reservation and rural districts, these 
sites do not have easy access to university training or staff support. Ihe on-site visits and state-wide 
meetings/conferences were viewed as extremely helpful. 

Program monitoring also proved to enhance program success; however, improvements to this 
process need to occur. There was initial uncertainty about the role of the external evaluator and the 
Department of Education in reference to annual reapprovals. Indeed, some of the information gathered 
by the external evaluators was also requested by the Department of Education and vice versa. Although 
individual evaluation reports prepared by Morrison Institute were available to ADE, this information was 
used inconsistently as part of the reapproval process. 

Most important, however, was the overall perception by districts that there were no real 
"consequences" for not implementing activities outlined in their proposals. While districts complied with 
evaluation requirements, they often did so in an untimely manner resulting in overall delays of the 
broader external evaluation conducted by Morrison Institute. In the two and one half years of Morrison 
Institute's involvement with the project, only one deadline was met by all districts-when ADE 
"threatened" to withhold annual reapprovals. Although several programs were placed on probation, most 
districts have been conditioned to believe that negative sanctions will not be enforced even if they are not 
in compliance. 

Historically, Department of Education efforts in reference to state-funded programs have focused 
primarily on monitoring budgets, not programs. As a result, the necessary program monitoring 
infrastructure is still being developed. Further, it is difficult to provide both technical assistance (e.g., 
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help a district in trouble) and enforce sanctions (e.g., withhold ftnding if improvement efforts are not 
being made), Perhaps these two tasks cannot be performed by the same entity. In sevt 1 states, a 
separate "accountability" office has been established to address this concern. Within Arizona, the 
Governor's Task Force has recommended that the Auditor General's Office take on the task of program 
monitoring. This separation of responsibilities should be given serious consideration. 

Findings suggest that expanded technical assistance be made available and that state monitoring 
continue. However, monitoring may need to be moved to a separate agency or department for the ex$)licit 
purpose of ensuring qualify compliance with funding requirements by enforcing sanctions when necessary. 

• The external evaluation process had a positive effect on program implementation; 
there is, however, a need for greater emphasis on district self-evaluation/analysis. 

Overall, the majority of districts indicated that the evaluation process ultimately benefitted the 
programs, although complying with such requirements was an extra burden on their staffs. Having an 
external evaluation prompted greater attention to implementation processes, services provided, students 
served, and desireu outcomes. District personnel, on the who! j, felt the external evaluation provided 
greater accountability and found the information provided to them important-both as a means to validate 
their efforts and as input for program improvement and local decision-making. In addition, the external 
evaluation has resulted in a wealth of general information for state policy makers. 

/ 

In efforts to determine the overall effectiveness of the total at-risk pi'ot project as well as of 
specific strategies, common data sets were collected to allow for individual program comparisons and 
aggregate analyses across all K-3 and all 7-12 programs. While this approach provided a wealth of data 
regarding the total project in a holistic sense, many of the unique characteristics and impacts of specific 
programs were "lost" in the aggregate. Although districts were encouraged to collect and report self- 
evaluation data, few districts did to. Instead, they relied solely on the external evaluation as their 
feedback loop. Many proguns did not, therefore, have their own unique data upon which to base 
refinement decisions. The external evaluation should not serve as a replacement for a systematic internal 
evaluation. Many districts acknowledged that they would have liked to conduct their own internal 
evaluations, but do not have the staff, time, or the expertise to do so. 

It appears that ttie externa! evaluation efforts have resulted in positive benefits at both the local 
and state level; such evaluation efforts should continue to be supported, but with additional emphasis on 
training district personnel to conduct quality studies of their own. 

• Valid measures of student outcomes and consistent recordkeeping and reporting 
mechanisms are lacking. 

The public is demanding the measurement and reporting of specific program outcomes. The 
development of appropriate assessment measures at the state level is highly desirable, particularly since 
many districts lack the expertise to define and measure program outcomes independently. Arizona has 
jumped ahead of most states through the development of its Arizona Student Assessment Program (ASAP). 
Unfortunately, this program was not available for use during this evaluation study. Indeed, on-going 
difficulties experienced in defining common outcome measures (in addition to standardized tests) lends 
support for the expeditious completion of the new assessment program. 

Within this evaluation project, desired outcomes among programs were identified and categorized 
so that common data sets could be collected across districts. However, determining adequate measures 
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of these outcomes proved to be extremely difficult. Lack of standardization existed among districts on 
variables such as absenteeism, dropout rate, and grade retention. Most districts had never collected data 
on parent involvement and staff training activities. Extensive efforts were necessary just to c^termine how 
many students, staff, and parents were served, with even greater energy required to measure the impact 
of these services. Data reporting standards had to be designed and then "taught" to the districts. The 
results often included calculation errors, requiring a tedious and lengthy verification process. Great gains 
in the measurement of program outcomes were achieved; further efforts to create comparable assessment 
measures (such as ASAP) will serve to enhance future external evaluations in this state. 

There is a need for continued/expanded support for the development of more standardized outcome 
measures. Districts and/or schools are ready to be held accou ltable, but most current assessment efforts 
and procedures are rudimentary at best and require continued state support and attention. 

• The label H at-risk H has, for many, negative connotations. 

Many pilot district personnel have expressed a dislike for the label "at-risk" because it is viewed 
as focusing on "the child as the problem." Educational leaders are increasingly advocating alternatives 
to the "at risk" label precisely because the label itself may connote underachievment and/or contribute 
to lowered expectations. For example, Levin (1990) has adopted the term "accelerated schools" to shift 
the emphasis away from the "child as the problem" to the "school as solution." 

A dislike for the term goes beyond a semantic debate: it has had visible consequences in terms 
of program implementation. For example, the labeling of funds, programs, and students as at-risk has 
contributed to poor communication within districts and between districts and communities. In some cases, 
there has been a reluctance to publicize that a district is the recipient of "at risk funds" for fear that 
parents, community members, and staff may find the term offensive. Many districts created their own 
program acronyms and were successful in communicating the goals and objectives of their at-risk 
programs; yet many others did not. 

The label has served a useful purpose in focusing attention on students at risk of academic failure 
because of poor environmental conditions, and in creating resources targeted toward these students. But 
the time is ripe for the state to provide leadership by adopting terminology that shifts the emphasis from 
"problem students" to school solutions. 

There should be some consideration of creating an alternative label to "at risk" at the state and 
local levels. 
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RECOMMENDATIONS 



Many suggestions are en edded in the previous sections of this chapter and this report contains 
many findings relevant to practice and policy at the local and state levels. This section synthesizes the 
findings in terms of key recommendations. It is by no means a comprehensive listing, but is intended to 
highlight state-level actions that could further the development of comprehensive programming for 
Arizona's students at risk. For the purpose of the recommendations, M state H is used when both legislative 
and Arizona Department of Education actions are encompassed within the recommendation. 



1. The Arizona Department of Education (ADE) should create a single, comprehensive unit 
which supports and assists local at-risk initiatives. 

By streamlining state and federally-funded program at-risk related personnel (e.g., K-3 At-Risk, 
7-12 At-Risk, LEP, Chapter 1, Migrant, substance abuse) into a single unit, the state can create 
the infrastructure and provide leadership to model and promote practices associated with 
successful program implementation at the local level. This is particularly needed in relation to 
"at-risk" districts and/or schools that do not have their own stable infrastructure. 

• Develop a comprehensive plan for meeting the needs of at-risk populations. 

Examine programs, funding goals and expectations, required paperwork, and existing 
accountability measures for the purpose of integrating these into a comprehensive plan. 

• Define general standards for local district/school recipients of at-risk funds, 
including expectations for local comprehensive plans to specify linkages among related 
programs (e.g., at-risk and LEP). 

• Specify criteria and measures for program accountability that meet federal and state 
requirements. Develop common criteria for non-test indicators (e.g., absenteeism, 
retention) since disparities currently exist in local definitions and reporting. 

• Coordinate/consolidate program reporting functions with the explicit goal of 
reducing paperwork. 

• Design a computerized database and recordkeeping system for compiling and 
analyzing outcome data (as determined by the comprehensive unit) and disseminate the 
technology state-wide. 

• Define and implement consequences for programs that either do not complete 
requirements or provide unusable information in meeting requirements to reinforce 
that the quality of programming, data collection, and reporting must be an integral part 
of compliance. 

• Coordinate technical assistance to at-risk populations/programs. 
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The legislature should establish permanent funding mechanisms to support at-risk programs 
at all grade levels, provided that the Arizona Department of Education has created the 
infrastructure to handle funding requirements and ensure local accountability. 

At-risk youth have shown progress within the context of the Arizona At-Risk Pilot Project. 
Continued support is required to serve additional students (e.g., grades 4-6) and allow programs 
to evolve and improve. Pilot project personnel have repeatedly expressed that "targeted" formula 
funding is preferable to both block and competitive grant funding. 

• Create an at-risk weight to provide basic targeted funding. Use factors such as 
poverty and mobility, as these indicators would provide a mechanism for directing funds 
into a district/school; local decisions would then be made as to which students are to be 
served. 

• Provide a small and rural school supplemental stipend, stipulating a base 
minimum. These schools typically have smaller enrollments that would limit an at-risk 
allocation based on formula funding, and therefore need additional resources to provide 
services as costly to implement as in larger districts (e.g., paying an additional teacher). 

• Fund educational services within county detention centers, using the Pima County 
Detention Center program as a model. 

• Institute an "education venture fund" with a separate request for proposal (RFP) 
process to encourage and reward innovative and comprehensive programs such as Family 
Resource Centers or alternative schools that require greater resources than provided by 
formula funding. Require that proposals are submitted jointly by a local educational 
agency in collaboration with a local business partner such as a social service agency, 
association, or other organization. 



The state should adopt an alternative label to "at risk." 

The adoption of an alternative label associated with funding (e.g., Comprehensive At Risk 
Education, or CARE, funding) could get a lot of mileage in terms of creating a positive "state 
climate" for further program development. Although the at-risk" label has been useful, it is 
generally perceived as a negative term when applied to children, their parents, and districts. 



Local districts and/or schools should develop comprehensive plans, aligned with state 
standards, as a condition for initial at-risk funding and reapproval. 

Local autonomy is essential in creating appropriate programs to match the needs of the local 
population. Evidence from this evaluation has shown, however, that local programs based on 
district/school plans have been more effective than those without such plans. 

• Require that plans describe how services targeting at-risk children are coordinated 
with related educational and social services. 

• Establish "planning grants" and/or allow adequate time to assist local districts 
develop their comprehensive plans. Based on feedback from pilot districts, good 
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planning that involves staff, parents, and/or community members, takes time-at least 
three to six months. 

• Provide individualized technical assistance upon request to facilitate the development 
of quality plans. 

5. The Arizona Department of Education's new "comprehensive at-risk education 11 unit should 
ensure that local plans are implemented. 

• Provide individualized technical assistance upon request to facilitate the 
implementation of quality programs. 

• Designate within ADE's comprehensive unit an "at-risk broker" who would link 
local programs to an identified state-wide network of technical assistance, and disseminate 
information about at-risk intervention strategies that have proven to be successful in 
model sites. 

• Monitor/audit programs annually at first, then on a two-three year cycle, to ensure 
progress toward locally-defined goals commensurate with state expectations. Consider 
separating this function from technical assistance. 

• Require districts to submit periodic self-evaluation reports (annually at first, then 
on a two-three year cycle). Program evaluation is essential for accountability and as a 
tool for program improvement. The state should develop model evaluation designs 
aligned with specific intervention strategies and other resources for use by district 
personnel. 

• Develop the expertise at ADE and throughout a state-wide technical assistance 
network to review district/school self-evaluation documents and compile quality 
outcome reports to assist in future decision-making. 

6. The Arizona Department of Education should use successful at-risk pilot sites to demonstrate 
"what works" for at-risk pupils. 

Pilot sites have invested four years in developing at-risk programs. Successful sites, and 
replicable programs (to be identified in Morrison Institute's forthcoming "what works" report), 
have staff who are valuable resources for sharing their expertise with ADE and other districts. 

• Contract with at-risk educators in successful programs to act as consultants/mentors 
regarding "promising practices' 1 for at-risk children and to provide peer training. 

• Subsidize training held "on-site" at these demonstration schools. 

7. The state should designate as its highest priority programs that specify improved language 
and literacy outcomes. 
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Extensive efforts have been expended to evaluate "what works" for Arizona's at-risk population. 
First and foremost, these efforts have corroborated 20 years of research that points to low 
language proficiency as a key correlate of academic failure, and low self-esteem as a by-product 
of such failure. Pilot sites have demonstrated progress toward improving students' language skills 
and self-esteem. The state should actively promote practices that have contributed to such 
progress. Promising practices include full-day kindergarten and reducing student-staff ratios at 
the K-3 level. At the 7-12 level, more "holistic" delivery systems (e.g., alternative schools) 
appear promising. In general: 

• Advocate increased individualization of instruction for at-risk pupils at all grade 
levels. 

• Expand the delivery of vocational/occupational education for older students, and 
the use of applied academics in particular. 

• Promote continued development of support services both on-site (e.g., counseling) 
as well as with community social service providers (e.g., Family Resource Centers). 



The state should develop mechanisms to assist local districts/schools improve parent 
involvement. 

• Reinforce requirements that K-3 districts/schools incorporate parent outreach as 
a condition for initial or continued funding. Virtually all K-3 personnel say that parent 
involvement is desirable. They also acknowledge that parent programs are difficult to 
implement, and that they might not have focused on parent services if not for the 
requirements of H.B. 2217 (1988). 

• Launch a public awareness campaign to encourage parents to support their 
children's education. 

• Allow expenditures for food purchases related to school social events (e.g., 
refreshments) to support district/school efforts in garnering public support. Latitude 
is required regarding what constitutes "acceptable" parent involvement activities. For 
many districts, efforts to attract parents to the school environment may be most effective 
when they involve non-academic activities. 

• Advocate secondary-level parent involvement programs. 

• Disseminate "promising practices" for increasing parent involvement including: 
workshops that actively involve parents, face-to-face communication (e.g., home visits), 
and model strategies used among 7-12 programs. 



Arizona's state university teacher training programs should be reconstructed with an 
emphasis on at-risk issues. 

Two issues are encompassed in this recomendation. First, there needs to be a greater emphasis 
on at-risk issues for all teachers, and a wider range of training opportunities. Second, there needs 
to be greater focus on recruiting and training minorities vo be teachers/administrators. 
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• Prepare all students in teacher training programs to work with students at risk. 

• Develop a curriculum for at-risk specialization. 

• Establish procedures for greater collaboration and articulation among schools and 
universities in planning and delivering pre-service programs. 

• Provide a wider range of pre-service training including: teaching experience with at- 
risk populations, internships with master teachers, and exposure to multicultural/ 
multilingual issues. 

• Use demonstration sites for student teaching and internships. 

• Review the effects of the Pre-Professional Skills Test (PPST), used as a 
requirement for admission into teacher training programs, on minority admissions. 

• Provide more training opportunities within rural and reservation communities to 
assist more local personnel in obtaining teaching and administrative certificate with a goal 
to reduce the "revolving door" syndrome apparent in these communities. 



State and local educational agencies should improve in-service training with respect to at- 
risk populations. 

• Use technology for in-service training delivery (e.g., workshops and college courses 
offered via satellite). 

• Establish procedures for greater collaboration and articulation among schools, 
community colleges, and universities in planning and delivering in-service programs. 

• Use demonstration site personnel as peer trainers for new district/school at-risk 
program staff. 

• Extend state efforts to provide in-service training by offering regularly scheduled at- 
risk conferences and/or academies, as these have been well -received. 

• Be a catalyst to influence local districts to reassess existing incentive programs for 

staff development, since some district policies regarding teacher compensation for 
advanced training are perceived as inadequate by the teaching staff. 



The Arizona Department of Education should waive annual at-risk funding reapprovals for 
current pilot sites that have demonstrated success. 

Pending reapproval of these programs in relation to a revised state comprehensive plan, some 
districts/schools should be eligible for continued funding and should immediately be placed on 
an extended evaluation cycle (i.e., every two-three years). At the discretion of the funding 
agency, however, other continuing districts/schools should be required to submit comprehensive 
plans and remain on an annual evaluation cycle. 
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Table Al 



1990-91 AT-RISK INDICATORS FOR K-3 STUDENTS BV REGION* 
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RURAL 




RESERVATION 




TOTAL 




At-Risk Indicator 


(n = 1336) 




(n = 1158) 




(n = 1 124) 






(n = 3618) 


(Questions 1 1-30, Student Profile) 
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Effect 
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Effect 
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43 1 




2.93 
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191 


Low annual income 
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Uv.4 


41 1 

41,1 


2.89 


^4 9 


J I.J 




58.1 


Jy.y 


Z.yo 


_> 2 schools attended 


31.0 


23.0 


3.07 


27 5 


17 9 


2.87 


17 1 
1 f . 1 


R 4 

0,** 


Z.O r 


25.6 


1a< A 
lo.o 


9 09 

Z.yZ 


Retained _> 1 time 


IS. 8 


8 4 


2.69 






2.54 


9n a 
zu.o 


O 4 


aV,J7 


16.4 


9 Q 

faO 


Z.Ol 


Low parent support 


43.5 


37 5 


3.26 


41 6 


11 3 

j j • j 


3.08 


£1 1 

O 1 • J 


41 SI 


1 OA 


48.7 


Ifl 1 
Jo. 1 


111 
J. 1 1 


Low parent participation 


57.4 


45.3 


3.15 


53 5 


39 8 


2.98 






9 07 


59.3 


41 A 


1 C\A 


Sibling dropout(s) 


6.4 


4 9 


3.07 


9 0 


1 5 


2.45 


1 1 9 


6 1 


? 70 


8.7 


*J.O 


9 99 

Z. /Z 


Low self-esteem 


48.7 


44 2 


3.33 


4S 6 


38 1 


3.15 


S4 R 


44 R 

•♦4.0 


1 14 


49.6 


49 

•♦Z.J 


1 91 


Chronic health problems 


18.4 


15.3 


3.16 


17.4 


12.6 


2.98 


22.2 


14.9 




19.3 


14.3 


3.04 


Substance abuse by child 


1.5 


1.0 


2.90 


4.2 


0.4 


2.14 


9.5 


5.7 


2.75 


4.9 


2.3 


2.60 


Substance abuse by parents 


13.8 


12.7 


3.41 


14.6 


10.4 


3.03 


28.8 


21.4 


3.13 
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16.5 
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Emot/behavioral problems 


31.1 


27.5 


3.25 


30.8T 


25.0 


3.06 


38.5 


30.0 


3.10 


33.3 


27.4 


3.14 


"Latch-key" situation 


19.2 


12.1 


3.12 


15.6 


9.8 


2.93 


25.6 


15.8 


2.86 


18.7 


12.5 


2.96 


Responsible for other siblings 


15.4 


10.5 


2.90 


13.9 


7.2 


2.71 


23.1 


11.8 


2.63 


17.6 


9.8 


2.74 


Transience/high mobility 


20.4 


17.4 


3.25 


19.2 


15.0 


2.97 


11.5 


5.8 


2.74 


17.2 


13.0 


3.04 


U.S. immigration (w/in 3 yrs) 


12.4 


9.5 


3.22 


18.3 


13.0 


2.99 


3.5 


0.5 


2.21 


11.8 


7.8 


3.01 


No/low level of English 


32.0 


24.3 


3.15 


43.5 


33.9 


3.05 


51.7 


38.3 


3.01 


41.8 


31.8 


3.06 


Poor parent-school contacts 


42.4 


35.6 


3.20 


42.8 


33.9 


3.10 


56.8 


36.8 


3.03 


47.1 


35.5 


3.11 


* % Applies » % population with indicator; % Affected » % population for whom indicator is believed to have some - a large negative effect 


on academic achievement; Mean Effect = degree of negative effect for all children to whom the indicator applies. 







SCALE: 2.00 =* Applies with NO negative effect on academic performance; 3.00 ■ Applies with SOME negative effect on academic performance; 

4.00 = Applies with LARGE negative effect on academic performance 
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Table A-2 



RANK ORDER OF INDICATORS BY ACADEMIC ACHIEVEMENT FOR K-3 AT-RISK STUDENTS (N = 3618)* 



Very High 
(n = 203) 



No indicstor(s) over 3.50 



Sibling dropout(s) 3.00 
Health problems 3.00 



ubstance abuse/PAR 2.89 
.Abusive home 2.86 
Emot/beh problems 2 . 78 
Parent-school contacts 2.71 
Few reading ma. ' his ^ .70 
Transience/mobility _67 
Substandard home 2.63 
"Latch-key" home 2.63 
Low self-esteem 2.61 
Low parent support 2 .60 
Low par. parth Ipation 2.58 
Low annual income 2.51 
Substance abuse/CH 2.50 



>2 schools attended 2.49 

Low Eng. proficiency 2.33 

Recent immigration 2.31 

Care for siblings 2.29 

Regained > 1 time 2.27 



High 
(ft = 503) 



No indicator(s) over 3 6 50 



No indicator(s) over 3.00 



Low self esteem 2.91 
Substance abuse/PAR 2.89 
Abusive home 2.86 
Emot/beh problems 2.79 
Few reading maieriab 2.74 
Parent-school contacts 2.70 
Low parent support 2.65 
Health problems 2.61 
Low par. participation 2.60 
2.56 
2.56 
2.56 
2.55 
2.50 



Substandard home 
Low annual income 
Low Eng. proficiency 
>2 schools attended 
Transience/mobility 



"Latch-key" home 2.48 

Recent immigration 2.47 

Care for siblings 2.42 

Substance abuse/CH 2.30 

Retained I lime 2.26 

Sibling dropout(s) 2.26 



Average 

(n = 1233) 



No indlcator(s) over 3.50 



Substance abuse/PAR 3.07 

Few reading materials 3.06 

Low self esteem 3 . 05 

Emot/beh problems 3.00 



Abusive home 2.99 

Parent-school contacts 2.93 

Low parent support 2 . 89 

Health problems 2.86 

Substandard home 2.84 

Low par. participation 2.83 

"Latch-key" home 2.83 

Low Eng. proficiency 2.83 

Low annual income 2.82 

Transience/mobility 2.8! 

Recent immigration 2 . 74 

.>. 2 schools ar >nded 2.69 

Care for siblings 2.63 

Sibling dropout(s) 2.60 



Substance abuse/CH 2.47 
Retained > 1 time 2.43 



Low 
(n = 1061) 



No ii iicator(s) over 3.50 



Few reading materials 3.28 

Low self esteem 3,27 

Substance abuse/PAR 3.24 

Recent immigration 3 .24 

Low parent support 3 ,22 

Abusive home 3.20 

Parent-school contacts 3.20 

Low par. participation 3.17 

Low Eng. proficiency 3.17 

Emot/beh problems 3.16 

Health problems 3.13 

Transience/mobility 3.13 

Low annual income 3.09 

"Latch-key * home 3.08 

Substandard home 3 .06 

> 2 scho* Is attended 3.05 



Sibling dropout(s) 2.87 

Care for siblings 2.84 

Substance abuse/CH 2.77 

Retained > 1 time 2.65 



No indicator(s) under 2.50 



Very Low 
(n = 608) 



Few reading materials 3.54 



Substance abuse/PAR 3.49 
Abusive home 3.49 
Low self esteem 3.44 
Low parent support 3 . 43 
Transience/mobility 3.41 
Low Eng. proficiency 3.40 
Emot/beh problems 3.39 
Low par. participation!.!! 
Parent-school contacts 3.37 
"Latch-key" home 3.34 
Recent immigration 3.31 
Substandard home 3.29 
Low am. »/ income 3.26 
>. 2 schools attended 3.25 
Health problems 3.25 
Care for siblings 3 ,06 



RetaincJ 1 time 2.94 
Sibling dropout(s) 2.90 
Substance abuse/CH 2.76 



No indicator(s) under 2.50 



* Table does not include 10 students whose academic pcrfoimance was not rated; Italics - indicators with trend across achievement ratings; Lines divide 
indicator* by range of effect (e.g., 2.0 - 2.49; 2.5 - 2.99) 

SCALE: 2.00 * Applies with NO negative effect on academic performance; 3.00 « Applies with SOME negative effect on academic performance; 4.00 
~ Applies with LARGE negative effect on academic performance 



Table A-3: K-3 STUDENT SERVICES/STRATEGIES (1990-91) 
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Table A-3: K-3 STUDENT SERVICES/STRATEGIES (1990-91) ~ continued 
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Table A-4: K-3 PARENT SERVICES/STRATEGIES (1990-91) 



K-3 Districts 
(Phase I) 




LEGEND D=District-Funded 



At-Risk Funded 



Table A-4: K-3 PARENT SERVICES/STRATEGIES (1990-91) - continued 
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Table A-5 
K-3 Student Participation Data* 



FY 1 990-9 j /Average Number Served 



District Region 


At-Risk Not At-Rlsk 


Total % At-Risk 


PHASE 1 






Ashfork 


2 


63 


0 


63 


100 


Chinle 


3 


61! 


778 


1.369 


44 


Coolidfc 


2 


43 


35 


78 


55 


Creighton 




340 


220 


560 


61 


G ana do 


3 


403 


165 


566 


71 


Kayenta 


3 


745 


0 


745 


100 


Laveen 
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0 
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100 


Littleton 




SSI 


0 


551 


100 


Mary C, O'Brien 




65 


0 


65 


100 


Morristown 




30 


0 


30 


100 


Murphy 
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0 


1.199 


100 


Nofatat 




465 


1,162 


1.647 
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396 
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56 
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297 
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965 
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Phoenix Elementary 
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3.246 
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Somerton 
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93 
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63 
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86 
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90 


Wilson 


1 
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426 
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TOTAL 




11,886 


4.016 


15.902 


75 


PHASE II 


Ajuila 


2 


77 
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69 


87 


Avondale 
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396 
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67 
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29 


62 


65 
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69 
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1,023 


319 


1.342 


76 


Dysart 




430 


430 


860 


100 


Eloy 




626 


33 


659 


95 


Fl Thomas 




138 


29 


167 


63 


Gadsden** 












Holbrook 




162 


136 


298 


54 


Hyder 




75 


0 


75 


100 


Isaac 




1.869 


599 


2,466 


76 


Peach Springs 




110 


0 


110 


100 


Red Mesa 




129 


0 


129 


100 


Salome 




33 


28 


61 


54 


Scanfteld 




205 


41 


246 


63 


Sunnyside 




I8l 


166 


367 


49 


Tempe 




m 


71 


309 


77 


Tuba City-Cameron 




93 


1 


94 


99 


Tuba City-Gap 




34 




35 


97 


TOTAL 




6,448 


- ISM 


6,836 


w 



* Based on district-reported numbers of students served 



** Offers a Summer School program only 



Table A-6 



DiltrKt 


RtfKHl 

Code 


Full day 
kindergarten 


Multi-year 
program 


Multi* 
grade 
program 


Other 
delivery 
tyitem 


Reduced 
student/staff 

rations: 
additional 


Reduced 
student/staff 
rations: 
additional 


Developmental^ 
appropriate 
practice! 


Student Sera 
Computer- 

aniited 
instruction 

integrated in 


Ugy 

Supplemenul 
individualised 
instruction/ 
tutoring 


Instructional 
support 
activities 


Instructional 
enrichment 
activitiei 


Additional/ 
expanded direct 
student service 
facility 


Additional/ 
expanded 
indirect 
student service 


On going 
student 
assessment 


On-going 
student 
assessment 


Additional/ 
expanded 
counseling/ 
social activities 


: ; i ; I . ; ; . ; . ; . 

Tool 


PHASE 1 






























AiMortt 


2 














• J 












41 








124 


CMnle 












It 


10 


102 




•nd 






•*d 


•nd 






SI 


$41 


Cootidge 


\ 


11 


41 








41 


41 




















142 


CfoJftitofl 












144 


10$ 


140 


















SI 


•41 


Can ado 














221 




mm 


IS 






IS! 


445 








1,014 


Kayonta 






102 


n 


IS 






SSI 






SSI 






SSI 








2414 


Laveen 




114 








Sfl 




S9I 






n":rm 






1,119 




21 


422 


1,944 


UttJo ton 


I 


121 








121 




512 




204| j 












Mary C. O'Brien 


2 




















u. — , ^ — ~+C0i 






45 






m 


197 


Horrlitown 




















10 
















20 


Murphy 


• 


324 












1,201 




















1,529 


NogalM 

Otborn 


• 












142 






















493 




1 


















jua 
JO 




27 


191 




44 




9 


421 
421 


Phoenln Elementary 


1 


212 








212 




212 






212 














IS! 


Pltacho 


1 












19 


It 


19 


IT 














• SOI 


Rooievetc 




M 






nam 




91 


SI4 






91 


91 




91 








1,124 


San Carlo* 












S)2 


$22 


511 


$32 










%ii 








' ) AAA 


Sanders 






I4S 


MS 


144 




121 


121 


120 


















1 111 


S ©merlon 
















4S2 




110 


1)1 


102 












MIS 


Whlterlver 










710 




17 


710 




42 




710 


17S 


710 


211 




all 


A 11 A 

ej it 


WHaon 




124 






n 




411 


424 










■ 424 


424 






ai 


■ l j in 
2,4 JO 


TOTAL I.OM 


2 10 


177 


1,247 


1,719 


2.454 


4,112 


1.211 


1,417 


2,170 


2.S19 


1,145 


4,211 


154 


204 


IJ45 


10,125 


PHASE II 






















Afulla 
















S9 










74 










111 


Avondalc 










14 


74S 


74S 






74 S 








745 


241 






3,137 


Balti 




17 








17 


17 


SI 






SI 


SI 








51 




101 


Buckeye 












WSSE$8i 








49 




_t* 






10 


41 


101 


234 


Douftju 




II 








mm 


1,192 


1,192 




SIS 






•nd 










4,012 


Dyiert 
















4)0| 4J0| 
















140 


Eloy 
























594 


457 










1,9)0 


ft- Thomas 












14 


21 












159 










251 


Gadsden" 




































0 


Holbrook 
















149 




14 
















211 


Hyder 
















71 






75 














ISO 


Nam 










220 


220 


220 


220 




lot 










115 






1,1)4 


Peach Spring i 
















110 




















110 


Rod Mtia 














129 


129 




















1SI 


Salome 










14 






10 






♦nd 














44 


Stanfitld 












40 




107 






19 


207 












413 


Sunnplda 










111 






140 






171 






r/i 








429 


Tampa 




14 








21 


14 


211 












•0 








410 


Tuba City «C amtron 
















91 






91 
















Tuba City-Cap 
















34 






14 














-3 


TOTAL 

- .. _l ..-1. .. 




• 4 


0 


0 


14) 


2.1)4 


2,167 


1,1* 1 


i.444 


1,412 


477 


2,144 


890 


1,02) 


426 


101 


101 


17,059 



9 

ERIC 



Table A-7 
K-3 Parent Participation Data 







At-Rifk Prof ram Stratecy 


uiscncc 


Region 
Code 


Enhanced 
written 
communication 


Enhanced verbal 
communication 


Home 

visits 


Take- 
home 

activities 
vW 

follow-up 


Formal 
parent 
volunteer 
programs 


Advisory 
roles for 
"at-risk" 
parents 


Events 


Parent 

workt Ho mi 


Structured 

Cap 

parents 


Additional / 

avni rw4ja/4 

counseling/ 
social services 


YoUl 


rrlAjfc 1 














Asnlork 


2 


78 




16 








62 


9 






I6S 


cntrfe 


; ■■■ 3 • 


1 '-166 


30 


6? 




13 




,. ' " 27 


161 








Loolidjc 


2 


172 


54 










76 


65 




1 


368 


Crdflhton 


1 




!;• 43 


n 






9 


153 


55 




ii 


• ;, :V--:'::,::k':::: ;r Jil 


Oanaoo 


J 
K 


488 














66 






~ 554 


Kiyent* 


J 






566 


466 
















Laveen 


i 
1 


797 


250 


IS 


40 


28 






■^nd 




iii 


1,263 


Littleton 


i" 

1 




1.010 




93 


452 


102 


*nd 


136 






♦nd 


1,812 


nary C O onen 


2 








4 








21 


13 




38 


Morrttt6wn 


• I" 


^ •".".•54 










7 


""55 








||9 


nurphy 


1 


1,203 


1,203 










245 


486 






3,137 


NofaJas 


• .': ; -2- 
















226 






226 


Osborn 


1 
1 




78 










220 


207 




9 


514 






232 


24 








5 




■: :M 




1* 


= :• 364 


Phoenix Elementary 


1 








212 








73 






285 


Pfcacho 


• 2 ■■■■ 




?v44 










v-::: 64 




"•,:•' —15 






Roosevelt 


1 


400 


98 


98 


98 






425 


154 




14 


L237 


San Cartel 




' : ' '""'111 




6 










V 35 






T5S 


Sanders 


3 








288 








49 






1 j37 


Soffiftftoti 


3 ■ 


486 






486 
















Whiteriver 


3 


555 


5SS 


296 


42 


54 


71 


937 


40 


7 


161 


2,718 


Wilson 




210 








10 




249 




■ 569 




1.038 










i,i/y 


i.vm 


717 


V2 


J.J9I 


rrsr 


67J 










0 




, . z ■ ,-. 


31 


• ; ,20 










. 33 




♦nd ,' 






Avondale 










550 


133 






40 






723 


BaJsi 


j ,: - 


v--. 65 


65 


II 




19 




59 


24 






243 


Buckeye 


i 


464 


61 


" 53 












22 


64 


664 


Bouflas 


J 


IJV 


,175 


27 










37 






366 


Dysart 


i 










ISO 




300 








450 


Boy 


i 


656 


4l4 


58 


IS 










40 


14 


iM) 


Ft. Thomas 


3 


83 












68 


12 






163 


Gadsden** 


. 1 






















0 


Holbrook 


2 


599 


8 




49 






SO 


16 






722 


Hyder 


2 


■ 9$ 




95 


64 






80 








334 


Isaac 


1 


193 


136 










42 








371 


Peach Springs 


3 








♦nd 








6S 






65 


Red Mesa 


i 


63 


17 








34 










114 


Salome 


2 






38 


n 






Jl 


20 






161 


Stanfield 


2 


250 


125 


61 


so 


38 




I7S 


32 






731 


Sunnyskk 


1 


H6 


85 




. 42 


34 




H 






24 


612 


Tempe 


1 


8S 


32 












15 






Ii2 


Tuba aty*Came/on 


3 


5S 


55 


16 


48 


4 






34 


3 


j 


218 


Tuba City- Gap 


3 


II 


II 


16 


42 


3 






12 


3 




101 


IUIAL 




7TT7 


DOT 


— 57J 


m 


JVJ 


7T 


B'41 JVV 


6B 


M2| 7.467 



9 

ERIC 



16 J 



BEST COPY AVAILABLE 



16 



Table A-8 

K-3 Staff Development Participation Data 
FY 1 990-9 1 Yearly Averz.ge 



Staff Development Strategy 








District 


code 


Workshops/ 
Inservices 


Conferences/ 
Academies 


Structured/ 
Formal 
Classes 


Program 
Meetings 


School Visits/ 
Observations 


Total 


rIAjc 1 












ASnlOrTC 


2 




3 


0 


0 


0 


6 


cronie 






6 


0 


3 


10 


170 




L 




19 


5 


7 


3 


165 




\ 


206 


14 


0 


236 


7 


463 


/-__ _ j_ 




271 


52 


9 


0 


14 


346 


Kaycnta ••: 




17 


5 


0 


0 


0 


22 




1 


108 


3 


0 


18 


0 


129 




| 




8 


6 


5 


3 


133 


Man/ f~ f*YRri*n 


2 


29 


4 


0 


22 


2 


57 




2 


19 


5 


0 


■ . ..0 


0 


24 


Murphy 


1 


448 


166 


10 


479 


26 


1,129 




• • 2 


532 


■■■ 9 


22 


16 


■:- 4 


583 


Iwvl 1 1 


I 


237 


4 


39 


59 


0 


339 




I ■■ 


21 


•:-^ : 4 


16 


■ 9 


1 


51 


Phoenix P|«mentarv 


1 


21 


0 


5 


26 


0 


52 




2 


5 


• 0 


. :: ■ 0 


. .0 


6 


II 


Roosevelt 


I 


64 


27 


0 


31 


0 


122 




I 


204 


19 


0 


67 


• 1 


291 


Sanders 


3 


107 


22 


19 


89 


3 


240 




2 


332 


46 


0 


23 


0 


401 




3 


208 


45 


32 


31 


1 


317 


Witson 


i 


66 


0 


6 


,,: 0 


0 


66 


TOTAL 




},i$i 


461 


165 


1,121 


81 


5,il7 


rnnjt ii 








0 




2 • 


10 


• • 1 


0 


25 


0 


3> 




I 


20 


0 


0 


55 


0 


75 


Ral«9 • 


,- 


7 


■ 4 




0 


4 


16 




2 


73 


0 


0 


3 


0 


76 




2 


737 


t70 


3 


95 


6 


1,011 


Dvsarc 


1 


135 


3 


0 


12 


0 


150 


Eloy 


2 


298 


53 


7 


0 


5 


363 


Ft Thomas 


3 


21 


2 


3 


8 


0 


34 


Gadsden"* 


2 












0 


Holbrook 


2 


29 




4 


89 


4 


137 


Hytfer 


2 


66 


22 


0 


29 


J 


i }n 

l iu 


Isaac 


1 


29 


6 


2 


20 


3 


60 


Peach Springs 


3 


304 


33 


18 


88 


5 


448 


Red Mesa 


3 


9 


4 


0 


35 


0 


48 


Salome 


2 


13 


A 


C 


3 


0 


20 


Sunfield 


2 


241 


2 


c 


1 


3 


247 


Sunnyside 


1 


IOC 


! 16 


13 


5 




138 


Tempe 


1 


8C 


> A 


3 


t 


) S 


> 96 


Tuba Oty-Cameron 


3 


IC 


) 3 


C 


> c 


) : 


16 


Tuba City-Gap 


3 


S 


> 3 


C 


) ( 


) : 


1 15 


TOTAL 




2.19 


54 


5' 


r Adi 


r 4; 


I 3.106 



" = Summer school only 



If?!; 



ERIC 
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BEST COPy mUBLE 



Table A-9 









K-3 INTERVIEWS 
FY 1990-91 










Parents 


Staff* 


School 


Community 1 


TOTALS 


District 




A ri mm 

AO null* 


Tchra 


Other 


Board 


Members 1 




Ash Fork 


4 


1 


5 


2 






12 


Chink 


3 


4 


13 


6 






26 


Coolidge 


4 


2 


6 








12 


Crcighton 


7 




8 


4 


1 


1 


23 


Ganado 


1 


5 


6 


6 


1 




19 


Kayenta 


3 


4 


10 


3 






20 


Laveen 


10 


3 


IS 


2 






30 


Littleton 


6 


3 


8 


1 


1 




19 


MC O'Brien 


6 


5 


4 


2 




2 


19 


Morristown 


6 


1 


4 


5 




2 


18 


Murphy 


6 


5 


5 








16 


Nogalea 


5 


7 


6 


3 


2 




23 


Osbom 


2 


2 


3 


1 






8 


Page 


1 


3 


11 


4 






19 


Phx. EL 


4 


3 


< 

j 




1 




13 


Picacho 


9 




K 

<j 


3 






19 


Roosevelt 


8 


12 


10 


6 




2 


38 


San Carlos 


2 


3 


5 


5 






13 


Sanders 


6 


4 


6 








16 


Somerton 


3 


3 


5 


4 


2 




17 


Whitcrivcr 


7 


3 


11 


3 


1 




25 


Wilson 


5 


3 


5 


2 






15 


TOTALS 


108 


80 


156 


62 


9 


7 


422 


Staff categories: Administrators = district administrators, school principals, program directors, etc.; 
Teachers = classroom teachers, certified program staff; Others = counselors, support staff, speciality staff, 
aides 



Table A-10 



1 K-3 STUDENT SERVICES EVALUATION BY REGION | 


URBAN/SUBURBAN 


RURAL 


RESERVATION 


Full day kindergarten 3.68 
Additional teachers) 3.66 
Summer lervicef 3, 54 
DAP/Curr. modifications 3.48 

1 Additional materials 3.48 
Additional aide(s) 3.43 

1 Enrichment activities 3.35 
During school tutoring 3.27 
Before/after school tutoring 3.16 

1 Computer use in classroom 3.14 
Improved facilities 3.04 
Multi-graded classroom(s) 2.99 
Counseling/psych, services 2.94 
Student assessment 2.93 

1 Multi-year classroom(s) 2.83 
OVERALL SERVICES 3.35 


Additional teachers) 3.58 
Full day kindergarten 3.42 
Additional materials 3.38 
DAP/Curr. modifications 3.37 
Summer services 3.36 
Additional aide(s) 3.32 
Enrichment activities 3.30 
Computer use in classroom 3.27 
During school tutoring 3.24 
Before/after school tutoring 3.13 
Multi-year classroom(s) 3.06 
Improved facilities 3.05 
Multi-graded claasroom(s) 3.03 
Student assessment 2.92 
Counseling/psych, sen .es 2.88 
OVERALL SERVICES 3 29 


Additional teachers) 3.54 
Full day kindergarten 3.47 
Additional aide(s) 3.45 
Additional materials 3.39 
Summer services 3.36 
Enrichment activities 3.32 
Computer use in classroom 3.24 
DAP/Curr. modifications 3.21 
During school tutoring 3. 16 
Before/after school tutoring 3.14 
Improved facilities 3.12 
Student assessment 2.89 
Multi-year classroom(s) 2.68 
Counseling/psych, services 2.55 
Multi-graded classroom(s) 2.54 
OVERALL SERVICES 3.21 


Scale: 1.0 = Hu not worked at all; 2.0 = Haa not worked well; 3.0 = Haa worked well; 4.0 = Haa worked very well 


| K-3 PARENT SERVICES EVALUATION BY REGION 


URBAN/SUBURBAN 


RURAL 


RESERVATION 


II More verbal communication 3.31 
II School oocial events J .26 
Parent workshops 3.16 
| More written communication 3.13 
Parent classes (e.g., GED/ESL) 3.06 
Formal home visits 3.03 
Take-home activities 3.03 
Parent volunteer program(s) 2.86 
Advisory roles for parents 2.80 
More counseling/psych, services 2.69 
OVERALL SERVICES 3.08 


School social events 3.35 
More verbal communication 3.28 
Parent classes (e.g., GED/ESL) 3.24 
Formal home visits 3.18 
Take-home activities 3. 14 
More written communication 3.12 
Advisory roles for parents 3.09 
Parent workshops 3.03 
More counseling/psych, services 2.87 
Parent volunteer program(s) 2.67 
OVERALL SERVICES 3.16 


School social events 3. 16 
Parent workshops 2.97 
Formal home visits 2.87 
More verbal communication 2.86 
Advisory roles for parents 2.76 
Take-home activities 2.75 
More written communication 2.73 
Parent classes (e.g., GED/ESL) 2.71 
Parent volunteer program(s) 2.68 
More counseling/psych, services 2.46 
OVERALL SERVICES 2.84 


| Scale: 1.0 = Has not worked at all; 2.0 - Has not worked well; 3.0 *= Hu worke 


d well; 4.0 «= Has worked very well 


1 K-3 STAFF 


SERVICES EVALUATION BY REGION 


URBAN/SUBURBAN 


RURAL J 


RESERVATION 


College/other classes 3.46 
ADE conferences/academies 3. 35 
On-site training/consultant 3.31 
Conference/Academics 3.27 
Regularly scheduled training 3.19 
Program planning meetings 3.15 
On-site training/district staff 3.12 
School visits/observation 3.08 
OVERALL SERVICES 3.25 


College/other classes 3.41 
ADE conferences/academies 3.36 
Conference/ Academics 3 . 34 
On-site training/consultant 3.23 
On-site training/district staff 3. 12 
School visits/observation 3.03 
Program planning meetings 3.00 
Regularly scheduled training 2.95 
OVERALL SERVICES 3.21 


On-site training/consultant 3.33 
ADE conferences/academies 3.26 
Conference/ Academies 3.26 
School visits/observation 3.08 
On-site training/district staff 3.07 
College/other classes 3.05 
Program planning meetings 3.05 
Regularly scheduled training 3.00 
OVERALL SERVICES 3.16 


|| Scale: 1 .0 = Has not worked at all; 2.0 = Has not worked well; 3.0 - Has worked well; 4.0 « Hu worked very well 


A-10 Hi) 
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7-12 SUPPLEMENTAL DATA 
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B 



Table B4: 7-12 STUDENT SERVICE DELIVERY SYSTEMS (1990-91) 1 



7-12 Districts 
(Phase I) 



Creighton 



Dyuit 



Ganado 



Alternative 
Schools 



Schools-within- 
schools 



CIasses/L«bs/Activities 



School- 
wide 
Reform 



Academic 



Vocational 



Support 



Kayenta 



Nog a lei 



Pinal County Consortium 
-Apache Junction 

-C*sa Grande Elem. 

-Caaa Grande UHS 
-Central AZ AJternative 

-Cooltdge Unified 
-Mammoth/San Manuel 
-Maricopa Unified 
-Superior Unified 
-Santa Cruz Valley UHS 



• X 



San Carlos 



Sanders 



Somerton 



Sunnyside 



Tucson 



7-12 Districts 
(Phase II) 



Marana 



Pima Co. Detention 
Center 



Primary Target Population: • = 7-8 grade component; X = 9-12 grade component 



D 



p » 

ERJ.C 



In several cimi, one symbol denotes more thin one program. 



B-l 



1G« 



7-12 STUDENT PARTICIPATION DATA 
FY 1990-91 



7-12 STUDENT PARTICIPATION 
BY DISTRICT 


FALL SE 
# served 


EMESTER 
# leaving 


SPRING I 
# served 


SEMESTER 
# leaving 


YEARLY 1 


TOTAL 


YEARLY 


AVERAGE 


PHASE 1 

CREIGHTON 

DYSART 

GANADO 

KAYENTA* 

NOGALES* 

PINAL COUNTY CONSORTIUM 

-Apache Junction Unified 
-Casa Grande Elementary 
-Casa Grande UHS 
-Central Arizona Alternative 
-Coolldce Unified 
-Mammoth/San Manuel Unified 
-Maricopa Unified 
-Santa Cruz Valley UHS 
-Superior Unified 

PINAL COUNTY TOTAL 
SAN CARLOS 
SANDERS 
SOMERTON** 
SUNNYSIDE 
TUCSON 

MARANA 
PIMA CO. 


49 
158 
120 
2466 
471 

181 
24 
1929 
116 
20 
15 
25 
55 

21 
2386 
360 

69 
281 
348 
420 

107 
487 


5 
44 
47 
120 
91 

152 
4 

257 
34 

2 
4 
3 
6 

P. 
462 

0 
13 

7 
107 
98 

43 
487 


72 
148 
154 
2466 
471 

199 
24 
1703 

125 
19 

i 12 
22 
65 

IS 
2187 
360 

51 
281 
449 
598 

123 
501 


2 
25 
77 
120 
91 

62 
1 

150 
36 
1 
3 
1 
11 

5 

270 
0 
20 
7 
63 
228 

78 
501 


# served 

121 
306 
274 
4932 
942 

380 
48 
3632 
241 
39 
27 
47 
120 
39 
4573 
720 
133 
569 
904 
1116 

230 
988 


fleaving 

7 
69 
124 
240 
182 

214 
5 

407 
70 
3 
7 
4 
17 

§ 
732 

0 

33 

14 

170 

326 

121 
988 


# served 

61 
153 
137 
2466 
471 

190 
24 
1816 

121 
20 
14 
24 
60 

2Q 
2287 
360 
67 
285 
452 
558 

1 1 5 
494 


# leaving 

4 
35 
62 
120 
91 

107 
3 

204 
35 
2 
4 
2 
9 

3 
366 

0 
17 

7 
85 
163 

61 
494 


TOTAL PHASE 1 & II 


7722 


1524 


7861 


1482 


15583 


3006 


7792 


1503 



*spr,ng semester figures based on fall data; no spring data available 

* 'based on spring data only dule to refinement In program services & reporting methods 



ounuuuuuMMUNl I Y LINKAGES: INSTRUCTIONAL SERVICES 



7-12 STUDENT PARTICIPATION 
BY DISTRICT 



PHASE I 

CREIGHTON 

DYSART 

GANADO** 

KAYENTA 

NOQALES 

PINAL COUNTY CONSORTIUM 
-Apache Junction Unified 
-Casa Grande Elementary 
-Casa Grande UHS 
-Central Arizona Alternative 
-Coolidge Unified 
-Mammoth/San Manuel 
-Maricopa 

-Santa Cruz Valley UHS 
-Superior 

PINAL COUNTY TOTAL 
SAN CARLOS 
SANDERS 
SOMERTON 
SUNNYSIDE 
TUCSON 

PHASE II 
MARANA 
PIMA CO. 



TOTAL 



CAI 1 CCI 

r ALL So 


flESTER 


SPRING SEMESTER 


YEARLY TOTAL 


YEARLY AVERAGE 


# 


# students 


# 


# students 


# 


# students 


# 


# students 


HDKages 


served 


linkages 


served 


linkages 


served 


linkages 


served 


1 


40 


■ 


1A 
1«» 


z 


54 


1 


27 


3 


28 


3 


37 


6 


65 


3 


33 


2 


67 


*nd 


•nd 


2 


67 


1 


34 


0 


0 


*nd 


•nd 


0 


0 


0 


0 


nd 


nd 


•nd 


•nd 


•nd 


•nd 


•nd 


*nd 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


2 


45 


2 


60 


4 


105 


2 


53 


nd 


*nd 


0 


0 


0 


0 


0 


0 


2 


20 


2 


36 


4 


56 


2 


28 


2 


13 


2 


21 


4 


34 


2 


17 


0 


0 


0 


0 


0 


0 


0 


0 


1 


55 


0 


0 


1 


55 


1 


28 


0 


0 


0 


0 


0 


0 


0 


0 


7 


133 


6 


117 


13 


250 


7 


125 


0 


0 


0 


0 


0 


0 


0 


0 




32 


1 


25 


2 


57 


1 


29 


0 


0 


*nd 


*nd 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


44 


1 


42 


2 


86 


1 


43 


0 


0 


2 


62 


2 


>2 


1 


31 


1 


487 


1 


501 


2 


988 


1 


494 


23 


831 


21 


798 


44 


1629 


22 


815 



*no data available 

* *no spring data available; yearly totals and averages are estimated 
based on fall data 
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SCHOOUCOMMUNITY LINKAGES: VOCATIONAL SERVICES 



2 



7-12 STUDENT PARTICIPATION 
BY DISTRICT 



PHASE I 
CREIGHTON 
DYSART 
GANADO 
KAYENTA 
NOGALES 

PINAL COUNTY CONSORTIUM 
-Apache Junction Unified 
-Case Grande Elementary 
-Casa Grand* UHS 
-Central Arizona Alternative 
-Coolldge Unified 
-Mammoth/San Manuel 
-Maricopa 

-Santa Cruz Valley UHS 
-Superior 

PINAL COUNTY TOTAL 
SAN CARLOS 
SANDERS 
SOMERTON 
SUNNYSIDE 
TUCSON 

PHASE II 
MARANA 
PIMA CO. 



OTAL 




'no data available 



cr 

ST 

W 



1 n 

1 i •* 



ERIC _ _ _ _ 



SCHOOL/COMMUNITY LINKAGES: SUPPORT SERVICES 



7-12 STUDENT PARTICIPATION 
BY DISTRICT 



PHASE I 

CREIGHTON 

DYSART 

QANADO 

KAYENTA 

NOGALES 

PINAL COUNTY CONSORTIUM 
-Apache Junctlci Unified 
-Casa Grande Elementary 
-Casa Grande UHS 
-Central Arizona Alternative 
-Coolidge Unified 
-Mammoth/San Manuel 
-Maricopa 

-Santa Cruz Valley UHS 
-Superior 

PINAL COUNTY TOTAL 
SAN CARLOS 
SANDERS 
SOMERTON 
SUNNYSIDE 
TUCSON 

PHASE II 
MARANA 
PIMA CO. 



TOTAL PHASE I & II 



FALL SEMESTER 



# 

linkages 



'no data available 



2 
14 
7 
3 

*nd 

12 
5 
0 
*nd 
6 
5 
1 
1 
6 

38 
9 
6 
0 
4 
3 



1 
4 



91 



# students 
served 



21 
308 
147 

20 
*nd 

984 
59 
0 
*nd 
28 
18 
25 
55 
49 
1218 
146 
159 
0 
5 
22 



4 
911 



SPRING SEMESTER 



2961 



# 

linkages 



1 
7 

5 

*nd 
*nd 

11 
4 

*nd 
4 
8 
4 
6 
1 
3 

41 
7 
1 

*nd 
0 
1 



0 
5 



# students 
served 



68 



2 
137 
42 
*nd 
*nd 

1015 
84 

*nd 

269 
34 
15 

198 
24 
24 
1663 

211 
25 

*nd 
0 
9 



0 

1059 



YEARLY TOTAL 



3148 



# 

linkages 



3 
21 
12 
3 
0 
0 

23 
9 
0 
4 

16 
9 
7 
2 
9 

79 

16 
7 
0 
4 
4 



1 

9 



159 



# students 
served 



23 
445 
189 
20 
0 
0 

1999 
143 
0 

269 
62 
33 
223 
79 
73 
2881 
357 
184 
0 
5 
31 



4 

1970 



6109 



YEARLY AVERAGE 



# 

linkages 



2 
11 
0 
2 
0 
0 

12 
5 
0 
2 
8 
5 
4 
1 
5 

40 
8 
4 
0 
2 
2 



1 

5 



# students 
served 



79 



12 
223 
95 
10 
0 
0 

1000 
72 
0 
135 
31 
17 
112 
40 
37 
1441 
179 
92 
0 
3 

16 



2 

985 



3055 



17', 
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Table B-6 



7-12 Parent Participation Data 
FY 1990-91 Yearly Average 



At-Risk Program Strategy 


District 


1A-W 


1A-V 


1B 


2C 


3A 


3B 


4 


Total 


















Phase 1 


















Creighton 


18 


14 


8 


4 


18 




,.3 


64 


Dysart 




77 


89 










166 


Ganado 


375 


7 


50 


62 


94 




108 


695 


Kayenta* 
















0 


Nogales 


189 


205 




, n 


111 






476 


Pinal Co. Consortium 
















0 


-Apache Junction 


125 


99 


5 










228 


-Casa Grande Elem 




12 


8 










20 


-Casa Grande UMS 








550 








550 


-C.A.A.S. 






15 


95 








109 


-Coolidge 


12 


8 


11 


7 








37 


-Mamrnoth/San Manual 




29 


3 


';' 7 


10 




5 


53 


-Maricopa 






28 


51 






8 


87 


-Santa Cruz 






8 


. .... 1 ■ 








9 


-Superior 




Aft 

26 












26 


San Carlos 


181 




97 


140 


24 




31 


471 


Sanders 






17 


13 


28 




21 


78 


Somerton 


464 






134 


88 






686 


Sunnyside 


170 


82 


13 


24 






7 


294 


Tucson 






20 


43 








63 


Total Phase 1 


1532 


557 


369 


1183 


288 


0 


182 


4109 


Phase II 


















Marana 




87 


1 










88 


Pima Co. Detention Ctr** 


















Total Phase II 




87 


1 










88 


Total 7-12 


1532 


731 


371 


1183 


288 


0 


182 


4285 



LEGEND NOTES 
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1A-W - Enhanced written communication 
1 A-V - Enhanced verbal communication 
1B - Home visits 

2A - Take-home activities w/ follow-up 
2B - Advisory roles for "at-risk* parents 
2C - Events 
3A - Parent workshops 
3B - Structured classes for parents 
4 - Additional/expanded counseling/social services 

B-6 



W^based on fall data only; 
activity did not occur in spring or 
data is incomplete/not available 
* 'program may not legally offer parent 
services; see program description for 
details 



7-12 Staff Development Participation Data 

FY 1990-91 Yearly Average 





: ;o/; : :.:v:v:v:v;v:v:vv:o:':v:v:v;v:::v:v;v:;v: 






^| , k^Iv^^^ , .^^^^v^^^.^ , iv^/!^^^■.',,■.^';■ 1 ^^v 










otatt uevelo 


)ment strate 






■':-x : : : ; : : : ;: : '': ; ; : : : ' : " : : : :-: : :: :: 


District 


Workshops/ 


Conferences/ 


Structured/ 


Program 


School Visits/ 


Total 




In-services 


Academies 


Formal Classes 


Meetings 


Observations 




Phase 1 














Creighton 


21 


17 


0 


0 


3 


41 


Dysart 


1 


5 


0 


15 


1 


21 


Ganado 


254 


44 


1 


0 


1 


299 


Kayenta* 












0 


Nogales 


8 


6 


1 


4 


0 


19 


Pinal Co. Consortium 












0 


-Apache Junction 


35 


20 


2 


16 


2 


73 


-Casa Grande Elem. 


9 


3 


2 


2 


1 


15 


-Casa Grande H.8. 


185 


14 


10 


8 


0 


217 


-Coolidge 


16 


0 


1 


16 


1 


33 


"Mammoth/San Manuel 


13 


8 


0 


26 


2 


48 


-Maricopa 


164 


1 


13 


9 


2 


188 


-Pinal Co. Alt. Ed. 


12 


7 


1 


23 


3 


45 


-Santa Cruz Valley 


2 


2 


0 


6 


0 


10 


-Superior 


10 


2 


0 


15 


2 


28 


San Carlos 


65 


0 


0 


39 


0 


104 


Sanders 


3 


4 


5 


0 


0 


11 


Somerton 


177 


3 


121 


18 


1 


320 


Sunnvside 


8 


4 


o 


130 


o 


142 


Tucson 


75 


19 


0 


13 


1 


107 


f III ..Phase, .l-pv 
Phase II 




'165 • 


■ 164.'; : 


■€,336: ' ; 


17 










Marana 


3 


5 


2 


8 


0 


18 


Pima Countv 


9 


7 


o 


49 


6 


104 


Total Phase ll 


• 12 


12 Ill 


| 2 


. 86 ; 






Total Phase I & II 


1068 


166 


166 


392 


23 


1958 



IViJ Ibi; 



Table B-8 



7-12 INTERVIEWS 
FY 1990-91 


District 


Parents 


Staff* 


School 
Board 


Community 
Members 


TOTALS 


Admin. 


Tchxs. 


Other 


Crcighton 


6 


2 


5 


2 


1 


2 


22 


Dygaxt 


18 


7 


13 


2 


1 


2 


49 


Ganado 


6 


5 


6 


4 


1 


- 


25 


Kayenta 


7 


2 


2 


3 


- 


- 


14 


Nogalea 


4 


4 


6 


5 


2 


- 


28 


Pinal County 

Cr% niAft 1 1 1 m 

VUI lavIllU III 


35 


7 


17 


12 




2 


83 


San Carlos 


4 


2 


1 


7 






14 


Sanders 


6 


2 


4 


3 






18 


Somerton 


3 


4 


4 


6 


2 




22 


Sunnyside 


3 


8 




4 


1 


3 


24 


Tucson 


6 


4 


6 


3 




2 


27 


TOTALS 


100 


45 


64 


51 


8 


11 


326 


Staff categories: Administrators — district 
Teachers = classroom teachers, certified 
aides 


administrators, school principals, program directors, etc.; 
program staff; Others = counselors, support staff, speciality staff, 



/A; 
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Table B-9 



7-12 PARENT SERVICES EVALUATION BY REGION 


URBAN/SUBURBAN 


RURAL 


Ktlot^K V A I IUIN 


School social events 2.80 
More verbal communication 2.72 
Formal home visits 2.66 
More written communication 2.54 
More counseling/psych, services 2.6S 
Parent classes (e.g., GED/ESL) 2.48 
Parent workshops 2.40 

OVERALL SERVICES 2.73 


More counseling/psych, services 2.77 
Parent classes (e.g., GED/ESL) 2.70 
Formal home visits 2.67 
More verbal communication 2.66 
School social events 2.66 
More written communication 2.56 
Parent workshops 2.17 

OVERALL SERVICES 2.65 


Formal home visits 2.72 
School social events 2.71 
Parent classes (e.g., GED/ESL) 2.55 
More verbal communication 2.46 
More written communication 2.45 
More counseling/psych, services 2.44 
Parent workshops 2.31 

OVERALL SERVICES 2.44 


Scale: 1.0 = Has not worked at all; 2.0 = Has not worked well; 3.0 = Has worked weil; 4.0 = Has worked very weU 



7-12 STAFF SERVICES EVALUATION B\ 


r REGION 


URBAN/SUBURBAN 


i 

RURAL 


RESERVATION 


Conferences/ Academies 2.81 
Program planning meetings 2.79 
On-site training/consultant 2.74 
Regularly scheduled training 2.72 
College/other classes 2.68 
School visits/observation 2.67 
On-site training/district staff 2.59 

OVERALL SERVICES 2.76 


Conferences/ Academies 2 .69 
Program planning meetings 2.68 
College/other classes 2.66 
On-site training/district staff 2.66 
Regularly scheduled training 2.64 
On-site training/consultant 2.60 
School visits/observation 2.45 

OVERALL SERVICES 2.70 


On-site training/consultant 2.60 
On-site training/district staff 2.54 
College/other classes 2.54 
Conferences/ Academics 2.50 
Program planning meetings 2.46 
Regularly scheduled training 2.30 
School visits/observation 2.29 

OVERALL SERVICES 2.58 


Scale: 1.0 = Has not vorked at all; 2.0 = Has not worked well; 3.0 = Has worked well; 4.0 = Has worked very well 
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APPENDIX C: 
BUDGET DATA 




PHASE I 



DiitHct 



Region 
Coda 



biitMt-iiveL 

ADMINISTRATION 



Table C I 

K-3 Pr ograms Budget Bre akdown by Function 

ScRSSCIRIl IbmtitituMMT [Ftm 



Tot^ 



ADHINISTIUTtON 



Toed 



SERVICES 



Tot* 



!?5T — 

ACTIVITIES 



Tot* 



(excluding Carry Forward) 

ifA^6tv«L6^HE^t — |N&4kAM £vAumtic 

| TRAINING 



Toed 



PROGRAM EVALUATION 



INDIRECt t6iU 



DISTRICT 
TOTAL 



Ashjork 
Chirie 



Co otkige 



Creighton 



Ganado 



lyenta 



17354 



0.00% 



0.00% 



0.00% 



0.00% 



2.54% 



24.48% 



0.00% 



0.00% 



IJOO 



mm 



m 

~~6" 



9.87% 



0.00% 



2.85% 



12.58% 



0.00% 



0.00% 



0.00% 



0.21% 



TSoo 



119.630 



I36.W1 



195.078 



J66.430 
30.700 



51.35% 



94.07% 



89.96% 



54.61% 



94.53% 



38.89% 



96.94% 



89.61% 



1356" 



mm 
mm 



MR 
73367 



mm 

25.100 



23.04% 



0.00% 



0.42% 



8.46% 



2.54% 



10.88% 



0.00% 



5.95% 



tZ400 



9.010 
1/556" 



"&" 



15.74% 



1.20% 



6.76% 



9.22% 



0.00% 



17.37% 



0.00% 



1.39% 



17J48 



0.00% 



0.00% 



0.00% 



15.13% 



0.00% 



8.38% 



0.00% 



0.00% 



12T 



4.73% 



0.00% 



0.00% 



0.40% 



200.657 



133,200 
249558 



206,367 



138? 



■jgaMfflata 



Mary C. O'Brien 



Morris town 



To 



0.00% 



0.00% 



0.04% 



7.08% 



33,020 



51.25% 



41.90% 



7350% 



T700 



451% 



1.00% 



2.34% 



0.00% 



200 



0.45% 



59,900 



0.00% 



8,056 



17.93% 



44,927 



Murphy 



0.00% 
0.00% 



0.00% 
0.00% 



248,443 
198,393 



97.85% 



589 



0.23% 



0.00% 



0.00% 



4,873 



1.92% 



253*904 



Nojaja 



Osborn 



0.00% 



Page, 



Phoenix Elementary 



4350 

—IT 



3.36% 



0.00% 



Picacho 



0.00% 



Roosevelt 



sir 



3.17% 
0.00% 



105,000 



100.00% 



0.00% 
3.97% 



112.991 
lOT 



83.33% 



0 

1356" 



0.00% 



0.00% 



2.78% 



0.00% 



94.96% 



San Carlos 



Sanders 



Whiceriver 



M32 
0 



4.66% 



0.00% 
0.00% 



1*240 



0.00% 



90.25% 



12100 



0.00% 



9.92% 



198,393 



1.000 



0.79% 



5.95% 



1OT 



17.41% 



222,862 

mm 



84.70% 



0.84% 



now 



0.00% 



126.000 



0.00% 



"Ml 



0.84% 



90.87% 



17.13% 



T2135T 



74.51% 



TJB55 

AM 



5.19% 



11400* 



3.41% 



0 



0.00% 



3.44% 



0.74% 



TO? 



0.00% 



J 18,991 



5.60% 



2.70% 



0.41% 



16358 



4.04% 



0.00% 



0.00% 

am 



o.oox 



145W 



70.09% 



100 



IflOO" 



2.71% 



000% 



IS? 



0.00% 



1.35% 



4.04% 



244,159 



0.34% 



0.00% 



15,542 



0.00% 



8.84% 



87.32% 



0.00% 



"TO6T 



81.41% 

"9TBW 



mm 

mm 



0.00% 



1.800 



1.08% 



6.329 



3.60% 



99.015 
175,648 



17.95% 



9,400 



5.65% 



975 



U3% 



0.64% 



0.00% 



"5755%" 



Agutla 



Data not submitted in time for analysis 



Balsz 



Buckeye 



Douglas 



0.00% 



o.oox 



0.00% 



5.40% 



5 



0.00% 



0.00% 



IJ3% 



0.00% 



r 4%5Sf 



93.38% 



97.07% 



68.16% 
89.84% 



^493 



2.92% 



0.00% 



12.14% 



0.00% 
127% 



m 



2.40% 



2.93% 



10.26% 



4.77% 



0.00% 



0.00% 
8.11% 



0.00% 



1.30% 



0.00% 



0.00% 



0.00% 



W>iT 



62.245 



Dysart 



0.00% 



0.32% 
0.84% 



"IT — 



95.95% 
59.66% 



155 
"TOT 



0,16% 



0.00% 



1.30% 



TT700 



Etoy 



29.400 



12.37% 



136? 



mm 

I 



6.70% 



14.75% 



0.00% 



im 



5.68% 



0.00% 



0.00% 
5.43% 



TOT 



93.28% 
6307% 



0.00% 



1300" 



672%" 



0.00% 



0.00% 



15JI? 



0.00% 



mm 

******** 

m 



m 



1.82% 



m 



1.79% 



1» 



0.72% 



mm 



27.17% 



Hyder 



IT 



7l4% 



0.00% 



Isaac 



0.00% 

~o!6o%" 



0.00% 



Dim* 



0.00% 
0.00% 



"TORT 
TOT 



77.58% 
51.04% 



96.67% 
26.82% 



0.00% 



34.84% 



0.00% 



1517%" 



IS556" 
lOT 



Tl4%" 



T 



1.29% 



0.00% 



7.98% 



0.00% 



1ST 



0.00% 



2.04% 



ignr 



Peach Sp' I np 



Red Mesa 



O00% 
0.00% 



0.00% 
0.00% 



TOT 
"W 



89.26% 
78.79% 



W5$ 



mm 



3.65% 



0.00% 



1557 
T3s7 



65.87% 



1856" 



3.65% 



7.1'% 



0.00% 



0.00% 



149% 



mm 



mm 



Stanfield 



Sunnyside 



Tcmpe 



Tuba City-Cameron 



Tuba City-Gap 



TOTAL 



136" 



10.05% 



Data not submitted 



4.45% 



6.70% 



~5!6o% 



0.00% 



1OT" 



T[ 

1538 



166.000 



0.00% 



0.00% 



24.09% 
"5537%" 



3.67% 



TOT 



c ; o.oo% 



109,727 



4.39% 



0.00% 



am 



100% 



53.919 



1. 71 



4.523,328 



TTTfi? 



83.78% 
59.75% 



82,65% 



3P&" 



156* 



2I7J032 



20.76% 



0.45% 



"lO^OO I '0.77% 

'Tol56%" 



3.97% 



nrooo* 

T556" 



274.134 
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1 8 ' 



3.38% 



9.46% 



5.39% 
1 



L03% 



5.01% 



Too 



75,252 



053% 



0.00% 



0.00% 

"6m 



IJ7% 



156" 



107,655 



4.02% 
1.91% 



0.00% 



92.822 



1.97% 



WIT 



5,473.127 



BEST COPY AVAILABLE 
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Table C-2 

K-3 Program Budgets without Carry Forward - By Region 







100th Day ADM 


Number 


Direct Student 


Cost/Pupil 


Total At-Risk 


Cost/Pupil 


Average 




Region 


K-3 Only 


Students 


Services 


Student 


Budget w/o 


Total 


Cost 


District 


Code 


1990-91 


Served 


$ 


Services 


Carry Forward 


Expenditures 


Per Pupil 


Avondale 


1 


1.263 


1.203 


143.811 


120 


154,004 


128 






1 


1.029 


62 


64365 


787 


66.51 1 


811 




Creighton 


1 


554 


560 


136.292 


243 


249358 


446 




Dysart 


1 


426 


430 


59,200 


138 


61,700 


143 




i>iac 


1 


2.-472 


2.468 


224,966 


91 


232.718 


94 




Laveen 


1 


798 


610 


158.397 


196 


163.391 


202 




Littleton 


1 


569 


551 


166.430 


302 


I8S.730 


337 




Murphy 


1 


1.023 


1,199 


246.443 


207 


253.904 


212 




Osborn 


1 


1.726 


909 


105.000 


116 


1 26,000 


139 




Phoenix Elementary 


1 


4.170 


212 


243.664 


1.149 


270.012 


1.274 




Roosevelt 


1 


4,902 


3.246 


222.862 


69 


245,259 


76 




Sunnyside 


1 


364 


367 


53.919 


147 


75,611 


206 




Tempe 


1 


305 


309 


74,366 


241 


88,760 


287 




Wjlion 


1 


420 


426 


262.903 


617 


277,193 


651 










$192 


Aguila 




Data not submitted in time for anal 


jrsis 




Ashfork 




67 


63 


7,800 


124 


15,191 


241 




Buckeye 




462 


242 


42.426 


175 


62,245 


257 




Coolidge 




883 


78 


119,630 


1336 


133,200 


1,708 




Douglas 




1 .349 


1.342 


317.898 


237 


353.864 


264 




Eloy 




658 


659 


141,603 


215 


237,692 


361 




Gadsden 




N/A 


134 


51,198 


382 


81.175 


606 




Holbrook 




490 


298 


97.359 


327 


125,488 


421 




Hyder 




72 


75 


27.73 1 


370 


54.332 


724 




Mary C O'Brien 




63 


65 


30.700 


472 


59,900 


922 




Morristown 




25 


30 


33.020 


1.101 


44,927 


1,498 




Nop las 




1.835 


1,647 


198.393 


120 


198,393 


120 




Picacho 




69 


69 


59,1 15 


664 


69,791 


784 




Salome 




56 


61 


i8,r>9 


299 


23,174 


380 




Somerton 




544 


545 


145.313 


267 


1 66,4 1 3 


305 




Stanfield 




Data not submitted 










$305 


Chinle 




1,414 


1.389 


188,757 


6 


200.657 


144 




Ft. Thomas 




166 


167 


48,616 


291 


52.116 


312 




Ganado 




674 


745 


195,078 


262 


206,367 


277 




Kayenta 




506 


566 


43.649 


77 


1 12.742 


198 




Pag« 




1.128 


985 


1 12.991 


115 


1 18,991 


121 




Peach Springs 




107 


no 


14.660 


133 


54,728 


498 




Red Mesa 




131 


129 


29,031 


225 


32.525 


252 




San Carlos 




598 


598 


73.775 


123 


99,015 


166 




Sanders 




321 


321 


123.259 


384 


175.848 


548 




Tuba City-Cameron 




95 


94 


55,464 


590 


92.822 


987 




Tuba City-Gap 




34 


35 


58393 


1.674 


99,412 


2,840 




Whiteriver 




868 


866 


123.551 


143 


151,766 


175 




liii mm iii 






mmmmmi 






$233 
$227 



N/A • Not Available 
l=Urban/Suburban 
2=Rural 
3= Reservation 
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Table C-3 



7-12 Programs Budget Breakdown by Function (excluding Carry Forward) 







DISTRICT-LEVEL 
ADMINISTRATION 


SCHOOL-LEVEL 
ADMINISTRATION 


DIRECT STUDENT 
SERVICES 


•ARENT ACTIVITIES 
ACTIVITIES 


STAFF DEVELOPMENT 
* TRAINING 


PROGRAM 
EVALUATION 


INDIRECT COSTS 


DISTRICT 
TOTAL 


District 


Region 
Code 


Total V 


Total X* 


Total X* 


ToUl X* 


ToUl X* 


Total %• 


Total %• 


Total 














Crcfehton 


1 




o.oox 




I9.I7X 


109.349 


6S.87X 


10.997 


6.6 2X 




3.3SX 


5.4*4 


3.30X 


" iW 


1.68% 


166*00 


Dysart 


1 


0 


0.00% 


32.719 


I8.03X 


119,90* 


66.0SX 


24.827 


I3.68X 


1.425 


0.79X 




O.OOX 


2.645 


1.46% 


MM 


Ganado 


3 


3^00 


2.4 IX 




O.OOX 


137.675 


92.IOX 


3,820 


25 6X 


3*65 


255% 




0.0OX 


590 


0.39% 






3 


0 


0.00% 


24 J 82 


I053X 


194.429 


84.68X 


0 


O.OOX 


8.000 


3.48X 


3.000 


1.3194 




0.00% 


iiWil 


Noples 


2 


0 


O.OOX 


25.658 


I0.7IX 


213.696 


89.20X 


0 


O.OOX 


i 225 


0.09X 




O.OOX 


0 


0.00% 


239.579 


Pinal Co. Consortium 


2 




2i8X 


-o 


O.OOX 


"i6S,>69 


93.24X 


5,693 


258X 


1,482 


0.67X 


2J04B 


0.93X 


0 


0.00% 


..''izOSfl 


San Carlos 


3 


■ ) 


O.OOX 




I0.82X 


mum 


76.36X 


iio6 


I.33X 


4.600 


4.08X 




0.0OX 


8,358 


7.41% 




Sanders 


3 




O.OOX 




I7.77X 


; I2i!i59 


72.7 IX 


600 


0.35X 


■""■'1^2 


9.I7X 




0.0OX 


" : : a 


0.00% 




Somerton 


2 




4.66% 


~m 


O.OOX 


mm 


90.7 IX 


"6 


O.OOX 


1.200 


0.56X 




4.07X 


"""""ft 


0.00% 




Stmnyside 


1 


2^00 


I57X 


0 


O.OOX 


153.480 


96.48X 


,.. m 


0.44X 


2.466 


151% 


- 0 


O.OOX 


0 


0.00% 


159.080 


Tucson 


1 


0 


O.OOX 


8,012 


ym 


177,422 


81.66* 


11.115 


5i8X 


2.731 


1.26* 


mil 


4.98X 


6,161 


2.84X 


MM 


PHASE II 












narana 


2 


0 


O.OOX 


0 


O.OOX 


154.392 


9I.62X 


0 


O.OOX 


614 


0.36X 


0 


0.00% 


13500 


8.0 IX 


HUM 


Pima County 


1 


m 




9,074 


I1.I6X 


65,144 


06.86% 


0 


o.oMi 


0 


6.00X 




0.06K 


6 


0.&6K 


•75.060 


TOTAL 






1,16% 


17? ''89 


/5*% 


1,933.530 


84.00% 


"60.355 


: . 2.62X 


L : : 47583 




; ; 30,019 




V 1 Wl 


I.48X 


2.301,705 



* Percent reflects percentage of Total FY 1 990/9 1 At-Risk Grant Budget 
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Table C-4 

7-12 Program Budgets without Carry Forward ■ By Region 



District 


Coot 


Numbtr 
Studtnts 
Strvtd 


Direct Student 
Services 

% 


Cost/Pupil 
Student 
Services 


Total At- Risk 
Budfet w/o 
Carry Forward 


Coat/Pupil 
Total 
Expenditure* 


Average 

Cost 
Per Pupil 


Crtijnton 


— ! 


77 


1 09,349 


1.420 


166.000 


2,156 




Opart 


— ! — - 


192 


1 1 A Oft 1 

1 19,901 


624 


181,517 


945 




Pima County 


— ! — 


AAA 

988 


65,144 


66 


75,000 


76 




Surmyikk 


- — ! — 


556 


153.480 


276 


159,080 


286 




Tucson 




696 


177.422 


255 


217,268 


312 
















$318 


narana 


— - — 


166 


154,392 


930 




1,015 




INOflMS 




471 


213.696 


454 


239,579 


509 




Pinal Co. Consortium 




2649 


205,769 


78 


220,691 


83 




Sonwton 




288 


192457 


670 


212,617 


738 












$235 


Ganado 




201 


137,675 


685 


149,490 


744 




Kajrtnta 




2466 


194,429 


79 


229.611 


93 




San Carlos 




360 


86.157 


239 


112,827 


313 




SanoVs 




64 


123.259 


1.926 


169,519 


2,649 








$214 




wmmmm 


mmnmmmmBmmmmmi\ $2$i 



I ■Urban/Suburban 
2*Rural 
3*IWsorvation 
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APPENDIX D: 
LIMITATIONS OF DATA SETS 




LIMITATIONS OF DATA SETS 



As noted in Chapter 2, the overall research design is conceptually qualitative and, as such, it is 
particularly subject to scrutiny relative to four criteria: credibility, transferability, dependability, and 
confirmability. Eveiy effort has been made to protect the study with respect to these criteria by 
continuously subjecting the methods and findings to internal and external review. Quantitative studies 
incorporated within the research have employed sampling methods, and results have been analyzed with 
respect to key variable potentially affecting the reliability and validity of the findings. We are confident 
in the study, as a whole, and believe it to be credible, transferable, dependable, and subject to 
confirmation. Yet, specific limitations exist with respect to specific databases, the most pertinent of which 
will now be addressed briefly. 

As reported for K-3 students, student profile data reflect the perceptions of K-3 teachers. 
Comparisons of 1989-90 and 1990-91 data lend some credence to the reliability of certain kinds of 
information reported, but also confirm the need for caution interpreting data as "fact. " These data are felt 
to be accurate and truthful reflections of teachers perceptions; and, preliminary comparisons of these data 
with available demographic databases tend to support the accuracy of teacher perceptions. 

Participation data for students, parents, and staff reported by district personnel, although 
confirmed and corrected by Morrison Institute staff, contain an unknown margin of error. Particularly, 
parent and staff numbers, as they are estimated and may reflect duplicated counts, should be used 
judiciously. 

All survey data (for 7-12 students and teachers, and K-3 teachers) have been considered 
regarding their "truth value." For teacher surveys, it is hypothesized that there is some degree of bias 
in overrating programs and program services. With respect to student survey data, it is believed that there 
may be some response bias in two mutually exclusive directions. First, some students may have 
underrated their "at-riskness" with respect to behaviors such as drug and alcohol abuse for fear of 
repercussions (even though the surveys were anonymous). Second, some students may have overestimated 
their at-riskness for "shock value." For example, it was brought to the attention of Morrison Institute staff 
that, in at least one case, students were told to exaggerate their responses in order to present a convincing 
picture of "at-riskness." Although such factors constitute threats to the validity of survey findings, it is 
felt that these threats are largely compensated for by sampling methods employed yielding large and 
proportionately representative survey responses. Suspected "data tampering" occurred among very small 
n's; other data biases may cancel out each other as they occur in both directions. 

Interview data utilized for this report reflect primarily those comments made during spring 1991 
site visits. On one hand, many of the interviewees were interviewed several times previously, leading to 
"diminishing returns" from the most recent interviews. Some local concerns were not repeated, as 
interviewees felt their site evaluators were already aware of them; likewise, the degree of program impact 
did not always seem as apparent given the history between the interviewee and site evaluator. On the 
other hand, in some school districts, there were many new staff interviewed because of high turnover. 
Comments from these staff members can not always be considered well grounded. Additionally, some 
parents and staff interviewed expressed fears that their funding might be jeopardized by negative 
comments and, therefore, may have emphasized only the most positive aspects of their programs. 
Moreover, because of the difficulty arranging parent interviews through appointments and home visits, 
the selection process unavoidably tended to favor those parents who are most involved in the schools, 
most aware of the at-risk program and its specifics, and, probably, most positive toward the school and 
program. In addition, the use of school officials as translators in some cases may have diluted responses. 
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Demographic and "impact" data (e.g., achievement test scores) gathered as part of the cohort 
study, while useful in presenting trends, should not be used to make inferences about at-risk program 
effectiveness. Cohort data are subject to virtually every threat to both internal and external validity. 
Student history and maturation are definite factors which contribute to the obtained results, as are changes 
in the nature of programs themselves as they have evolved over time. One other caveat concerns the 
interpretation of low n's for students included in the longitudinal tracking efforts. There is some 
propensity to associate low n's with student mobility. While student mobility is indeed a factor that 
accounts for some reduction in student numbers, there is also the problem of data retrieval. Much 
information that is missing, preventing a student from being included in the long'tudinal analysis, is a 
result of problems obtaining or reporting data-not a result of student mobility. 

Budget information, used in deriving cost-per-pupil estimates, should be viewed with some 
caution as well. Although operational definitions and detailed instructions were provided to each district, 
it is likely (based on the number and types of district questions received by Morrison Institute) that not 
all district reported expenditures from a common point of view. This makes the comparability of data 
reported subject to question. Further, it is clear that the expertise of the person completing the budget 
forms may have affected the quality of data submitted. In some cases, the budget manager completed the 
forms; in other cases, it may have been a program secretary, aide, or volunteer who may not always have 
understood the nuances of reporting budget information in the manner requested. Another factor that 
should be considered when viewing the budget data is that some program participants, aware of the fact 
that the information is being used to determine future funding, may have skewed their expenditures 
toward the category of "direct student services." 

While individual databases are subject to certain threats of validity, and may not be generalizable 
beyond the scope of the current at-risk pilot sites, the strength of this project rests in its magnitude 
combined with the fact that no one database takes precedence over another. The factors, programs, or 
practices that emerge as themes as a result of the analysis may be posited as generally useful for 
consideration hi other settings. 
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Morrison Institute for Public Policy 



Established in 1981 through a gift from the Morrison family of Gilbert, Arizona, Morrison 
Institute for Public Policy is an Arizona State University (ASU) resource for public policy research, 
expertise, and insight. The Institute conducts research on public policy matters, informs policy 
makers and the public about issues of importance to Arizona, and advises leaders on choices and 
actions. A center in the School of Public Affairs (College of Public Programs), Morrison Institute 
helps make ASU s resources accessible by bridging the gap between the worlds of scholarship and 
public policy, 

The Institutes primary functions are to offer a variety of services to public and private sector 
clients and to pursue its own research agenda. Morrison Institute's services include policy research 
and analysis, program evaluation, strategic planning, public policy forums, and support of citizen 
participation in public affairs. The Institute also serves ASU's administration by conducting 
research pertinent to a variety of university affairs. 

Morrison Institutes researchers are some of Arizona's most experienced and well known policy 
analysts. Their wide-ranging experiences in the public and private sectors and in policy develop- 
ment at the local, state, and national levels ensure that Morrison Institute's work is balanced and 
realistic. The Institute's interests and expertise span such areas as education, urban growth, the 
environment, human services, and economic development 

The Institute's funding comes from grants and contracts from local, state, and federal agencies 
and private sources. State appropriations to Arizona State University and endowment income 
enable the Institute to conduct independent research and to provide some services pro bono. 



Morrison Institute for Public Policy 
School ol Public Affairs 
Arizona State University 
Tempe, Arizona 85287-4405 
(602) 965-4525 
(602) 965-9219 (fax) 
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